2024-04-23T18:01:34.962 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T18:01:34.962 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102 --verbose --owner scheduled_leonidus@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} -- /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/orig.config.yaml 2024-04-23T18:01:35.024 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T18:01:35.128 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7670102' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5182 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGineUW3WTMQEPPiuKNhabBgQQ5rdA5pJPqvef+iVH2nvXESQ3TWLfqACU/i0w0SDzyRfY5gtyUU2nFEz9E49dM= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgZ6OW/XO2bJf/adqJ+8TNFlf9ZNyrFxFkFUyeYJV/xhNJl4Ry8pVDYYdidBqV1mgYTt7iwsZ3FDWcrouy59f8= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEEKfLdJDOFNpjNjouRgn/SwbffJWlDEJkZnanyJGJ5ZpbJ8FbcQbFlY/a2uFQErQtVHZ6747PhSYVtM0YRpfU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_17:44:29 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T18:01:35.129 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa; will attempt to use it 2024-04-23T18:01:35.129 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks 2024-04-23T18:01:35.131 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T18:01:35.133 INFO:teuthology.task.internal:Checking packages... 2024-04-23T18:01:35.157 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-23T18:01:35.157 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T18:01:35.157 INFO:teuthology.packaging:ref: None 2024-04-23T18:01:35.157 INFO:teuthology.packaging:tag: None 2024-04-23T18:01:35.157 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-23T18:01:35.157 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:01:35.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-23T18:01:35.327 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-23T18:01:35.328 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T18:01:35.335 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T18:01:35.335 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T18:01:35.356 INFO:teuthology.task.internal:Saving configuration 2024-04-23T18:01:35.378 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T18:01:35.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:01:35.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 17:56:22.629390', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGineUW3WTMQEPPiuKNhabBgQQ5rdA5pJPqvef+iVH2nvXESQ3TWLfqACU/i0w0SDzyRfY5gtyUU2nFEz9E49dM='} 2024-04-23T18:01:35.434 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 17:56:22.627461', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgZ6OW/XO2bJf/adqJ+8TNFlf9ZNyrFxFkFUyeYJV/xhNJl4Ry8pVDYYdidBqV1mgYTt7iwsZ3FDWcrouy59f8='} 2024-04-23T18:01:35.458 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 17:56:22.630953', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEEKfLdJDOFNpjNjouRgn/SwbffJWlDEJkZnanyJGJ5ZpbJ8FbcQbFlY/a2uFQErQtVHZ6747PhSYVtM0YRpfU='} 2024-04-23T18:01:35.458 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T18:01:35.464 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-23T18:01:35.465 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-23T18:01:35.465 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-23T18:01:35.465 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T18:01:35.574 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd57fc500d0>, signals=[15]) 2024-04-23T18:01:35.575 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T18:01:35.583 INFO:teuthology.task.internal:Opening connections... 2024-04-23T18:01:35.583 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:01:35.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:01:35.669 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-04-23T18:01:35.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:01:35.744 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-04-23T18:01:35.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:01:35.820 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T18:01:35.826 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-23T18:01:35.845 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-23T18:01:35.845 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:01:35.902 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:01:35.903 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:01:35.903 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:01:35.903 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:01:35.903 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:01:35.903 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-23T18:01:35.925 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-04-23T18:01:35.943 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-04-23T18:01:35.943 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:01:36.001 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:01:36.002 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:01:36.002 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:01:36.002 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:01:36.002 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:01:36.002 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-04-23T18:01:36.025 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-04-23T18:01:36.043 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-04-23T18:01:36.043 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:01:36.100 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:01:36.101 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:01:36.101 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:01:36.101 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-04-23T18:01:36.125 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T18:01:36.135 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T18:01:36.141 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T18:01:36.141 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:01:36.144 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:01:36.146 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:01:36.162 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T18:01:36.168 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T18:01:36.168 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:01:36.201 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:01:36.203 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:01:36.318 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T18:01:36.396 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T18:01:36.414 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-23T18:01:36.414 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-23T18:01:36.414 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-23T18:01:36.415 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:01:36.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:01:36.432 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-23T18:01:36.491 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-23T18:01:36.492 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-04-23T18:01:36.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-04-23T18:01:48.873 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 759 kB/s | 8.0 MB 00:10 2024-04-23T18:02:11.505 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 905 kB/s | 19 MB 00:21 2024-04-23T18:02:15.638 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-23T18:02:22.109 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 3.1 kB/s | 16 kB 00:05 2024-04-23T18:02:23.674 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-23T18:02:28.933 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-23T18:02:30.881 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T18:02:30.881 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T18:02:30.961 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-23T18:02:30.962 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-23T18:02:30.962 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-23T18:02:31.084 DEBUG:teuthology.orchestra.run.smithi037:> echo no | sudo yum reinstall kernel || true 2024-04-23T18:02:31.604 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 06:02:28 PM UTC. 2024-04-23T18:02:31.727 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-23T18:02:31.740 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-23T18:02:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-23T18:02:31.740 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-23T18:02:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:Operation aborted. 2024-04-23T18:02:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:Is this ok [y/N]: 2024-04-23T18:02:31.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum reinstall -y kernel || true 2024-04-23T18:02:32.309 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:04 ago on Tue 23 Apr 2024 06:02:28 PM UTC. 2024-04-23T18:02:32.431 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-23T18:02:32.431 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-23T18:02:32.431 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-23T18:02:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-23T18:03:18.031 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64.rpm 437 kB/s | 255 kB 00:00 2024-04-23T18:03:18.031 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:03:18.031 INFO:teuthology.orchestra.run.smithi037.stdout:Total 5.6 kB/s | 255 kB 00:45 2024-04-23T18:03:18.049 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-23T18:03:18.251 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-23T18:03:18.252 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-23T18:03:18.257 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-23T18:03:18.258 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-23T18:03:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-23T18:03:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:03:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:03:18.786 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:03:18.787 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:03:19.479 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:03:19.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:03:19.480 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalled: 2024-04-23T18:03:19.480 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T18:03:19.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:03:19.480 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-23T18:03:19.642 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T18:03:19.691 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T18:03:19.691 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T18:03:19.692 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T18:03:19.692 DEBUG:teuthology.parallel:result is None 2024-04-23T18:03:19.692 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T18:03:19.702 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T18:03:19.703 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:03:19.706 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:03:19.708 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:03:19.726 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T18:03:19.733 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T18:03:19.742 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T18:03:19.743 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:03:19.761 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:03:19.766 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:03:19.870 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T18:03:19.877 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T18:03:19.877 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:03:19.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:03:19.894 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:03:19.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:03:19.911 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:03:19.913 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:03:19.937 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:03:19.959 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:03:19.974 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:03:19.977 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:03:19.996 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:03:20.015 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:03:20.018 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:03:20.020 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T18:03:20.031 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T18:03:20.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:03:20.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:03:20.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:03:20.089 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T18:03:20.100 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T18:03:20.100 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:03:20.103 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:03:20.128 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:03:20.144 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:03:20.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:03:20.244 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:03:20.328 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:03:20.365 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:03:20.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:03:20.429 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:03:20.472 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:03:20.520 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:03:20.604 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:03:20.647 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:03:20.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:03:20.713 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:03:20.759 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:03:20.812 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:03:20.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:03:20.939 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:03:20.939 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:03:21.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-23T18:03:21.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-04-23T18:03:21.011 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-04-23T18:03:21.043 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:03:21.051 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:03:21.083 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:03:21.412 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T18:03:21.420 INFO:teuthology.task.internal:Starting timer... 2024-04-23T18:03:21.420 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T18:03:21.431 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T18:03:21.439 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-23T18:03:21.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-23T18:03:21.523 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-23T18:03:21.525 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-04-23T18:03:21.609 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-04-23T18:03:21.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-04-23T18:03:21.682 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-04-23T18:03:21.685 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T18:03:21.685 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-23T18:03:21.711 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-23T18:03:21.711 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-04-23T18:03:21.733 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-04-23T18:03:21.733 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-04-23T18:03:21.757 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-04-23T18:03:21.758 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2024-04-23T18:03:21.758 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:03:21.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:03:21.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:03:21.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:03:21.812 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:03:21.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:03:21.840 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T18:03:21.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:03:21.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:03:21.892 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:03:21.926 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T18:03:21.936 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T18:03:21.955 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T18:03:21.957 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi172.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T18:07:19.220 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-04-23T18:07:19.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-23T18:07:19.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:07:19.304 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-23T18:07:19.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-23T18:07:19.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-23T18:07:19.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:07:19.463 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-23T18:07:19.547 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-23T18:07:19.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-23T18:07:19.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:07:19.623 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-04-23T18:07:19.703 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-23T18:07:19.703 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T18:07:19.716 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T18:07:19.717 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:07:19.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:07:19.720 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:07:19.720 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:07:19.722 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:07:19.722 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:07:19.749 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:07:19.750 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:07:19.763 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:07:19.765 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:07:19.773 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:07:19.786 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-04-23T18:07:19.787 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:07:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-23T18:07:19.797 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-04-23T18:07:19.803 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-23T18:07:19.810 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:07:19.811 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-04-23T18:07:19.817 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:07:19.822 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-04-23T18:07:19.823 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:07:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:07:19.835 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:07:19.848 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:07:19.879 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:07:19.882 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:07:19.882 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-23T18:07:19.883 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.883 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.883 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.883 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:07:19.889 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:07:19.889 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-23T18:07:19.889 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.889 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.889 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.889 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.902 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:07:19.903 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:07:19.904 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-23T18:07:19.904 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.904 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.904 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.904 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:07:19.904 INFO:teuthology.run_tasks:Running task install... 2024-04-23T18:07:19.914 DEBUG:teuthology.task.install:project ceph 2024-04-23T18:07:19.914 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-23T18:07:19.915 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-23T18:07:19.915 INFO:teuthology.task.install:Using flavor: default 2024-04-23T18:07:19.922 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T18:07:19.923 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T18:07:19.923 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:07:19.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:07:19.926 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:07:19.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:07:19.927 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:07:19.927 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:07:20.189 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:07:20.189 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:07:20.200 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:07:20.200 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:07:20.216 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:07:20.216 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:07:20.526 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:07:20.527 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:07:20.527 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:07:20.552 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:07:20.553 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:07:20.584 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:07:20.584 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:07:20.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:07:20.608 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:07:20.608 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:07:20.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:07:20.614 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:07:20.615 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:07:20.617 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:07:20.636 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:07:20.637 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:07:20.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:07:20.690 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:07:20.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:07:20.753 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-04-23T18:07:20.755 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-04-23T18:07:20.764 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:07:20.778 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:07:20.831 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-23T18:07:20.833 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-23T18:07:20.844 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-04-23T18:07:20.845 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-04-23T18:07:20.981 INFO:teuthology.orchestra.run.smithi173.stdout:48 files removed 2024-04-23T18:07:21.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:07:21.072 INFO:teuthology.orchestra.run.smithi172.stdout:48 files removed 2024-04-23T18:07:21.073 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-23T18:07:21.101 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:07:21.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:07:22.164 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 93 kB/s | 83 kB 00:00 2024-04-23T18:07:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 87 kB/s | 83 kB 00:00 2024-04-23T18:07:22.390 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 81 kB/s | 83 kB 00:01 2024-04-23T18:07:22.740 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-23T18:07:22.789 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-23T18:07:22.790 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-23T18:07:23.315 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-04-23T18:07:23.457 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-23T18:07:23.457 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-23T18:07:24.265 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-23T18:07:24.482 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-23T18:07:26.725 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-23T18:07:28.000 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-04-23T18:07:31.067 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-23T18:07:32.481 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 917 kB/s | 8.0 MB 00:08 2024-04-23T18:07:32.936 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-23T18:07:33.034 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-04-23T18:07:34.905 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-23T18:07:35.612 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-23T18:07:37.603 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-23T18:07:40.187 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-23T18:07:42.497 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:07:42.497 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:07:42.510 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:07:42.510 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:07:42.511 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:07:42.512 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:07:42.514 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:07:42.516 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:07:42.517 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:07:42.584 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-23T18:07:42.593 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-23T18:07:42.593 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:07:42.594 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:07:42.595 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:07:42.596 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:07:42.597 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:07:42.598 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:07:42.599 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:07:42.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:07:42.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:07:42.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:07:42.603 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:07:42.604 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:07:42.605 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:07:42.606 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-23T18:07:42.606 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-23T18:07:42.606 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-23T18:07:42.606 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-23T18:07:42.606 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:07:42.607 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-23T18:07:42.607 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-23T18:07:42.637 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-23T18:07:43.554 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-23T18:07:43.771 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 4.0 MB/s | 890 kB 00:00 2024-04-23T18:07:43.838 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-23T18:07:44.139 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 7.0 MB/s | 5.4 MB 00:00 2024-04-23T18:07:44.181 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-23T18:07:44.306 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 9.0 MB/s | 1.5 MB 00:00 2024-04-23T18:07:44.715 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 8.8 MB/s | 4.7 MB 00:00 2024-04-23T18:07:44.737 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:07:44.738 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:07:44.748 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:07:44.749 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:07:44.750 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:07:44.751 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:07:44.752 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:07:44.754 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:07:44.755 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:07:44.815 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-23T18:07:44.834 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-23T18:07:44.834 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-23T18:07:44.834 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-23T18:07:44.834 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-23T18:07:44.834 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:07:44.835 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:07:44.836 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:07:44.837 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:07:44.838 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:07:44.839 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:07:44.840 INFO:teuthology.orchestra.run.smithi173.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:07:44.841 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:07:44.842 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:07:44.843 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:07:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:07:44.845 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:07:44.846 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout:Install 148 Packages 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 4 Packages 2024-04-23T18:07:44.847 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:07:44.848 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 222 M 2024-04-23T18:07:44.848 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-23T18:07:45.739 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 12 MB/s | 17 MB 00:01 2024-04-23T18:07:45.806 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 378 kB/s | 25 kB 00:00 2024-04-23T18:07:45.812 INFO:teuthology.orchestra.run.smithi173.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-23T18:07:46.032 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 8.0 MB/s | 11 MB 00:01 2024-04-23T18:07:46.132 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-23T18:07:46.208 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 360 kB/s | 27 kB 00:00 2024-04-23T18:07:46.283 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-23T18:07:46.575 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): ceph-common-19.0.0-3156.gd69f13d8.el7 6.6 MB/s | 21 MB 00:03 2024-04-23T18:07:46.600 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 747 kB 00:00 2024-04-23T18:07:46.634 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-23T18:07:46.659 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 2.1 MB/s | 126 kB 00:00 2024-04-23T18:07:46.704 INFO:teuthology.orchestra.run.smithi173.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 998 kB/s | 890 kB 00:00 2024-04-23T18:07:46.792 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 3.1 MB/s | 500 kB 00:00 2024-04-23T18:07:46.863 INFO:teuthology.orchestra.run.smithi173.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 915 kB/s | 145 kB 00:00 2024-04-23T18:07:46.901 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 416 kB/s | 45 kB 00:00 2024-04-23T18:07:47.010 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.2 MB/s | 135 kB 00:00 2024-04-23T18:07:47.127 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.4 MB/s | 161 kB 00:00 2024-04-23T18:07:47.252 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.5 MB/s | 320 kB 00:00 2024-04-23T18:07:47.330 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-04-23T18:07:47.386 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.4 MB/s | 5.4 MB 00:00 2024-04-23T18:07:47.411 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 298 kB 00:00 2024-04-23T18:07:47.461 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-23T18:07:47.503 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 929 kB/s | 85 kB 00:00 2024-04-23T18:07:47.611 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-23T18:07:47.729 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 293 kB/s | 34 kB 00:00 2024-04-23T18:07:47.856 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.1 MB/s | 142 kB 00:00 2024-04-23T18:07:47.913 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 6.8 MB/s | 3.1 MB 00:00 2024-04-23T18:07:48.238 INFO:teuthology.orchestra.run.smithi173.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-23T18:07:48.288 INFO:teuthology.orchestra.run.smithi173.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-23T18:07:48.382 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 6.7 MB/s | 3.5 MB 00:00 2024-04-23T18:07:48.498 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.1 MB/s | 249 kB 00:00 2024-04-23T18:07:48.599 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 493 kB/s | 49 kB 00:00 2024-04-23T18:07:48.699 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 165 kB/s | 16 kB 00:00 2024-04-23T18:07:48.833 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.0 MB/s | 273 kB 00:00 2024-04-23T18:07:49.022 INFO:teuthology.orchestra.run.smithi173.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-23T18:07:49.083 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.0 MB/s | 762 kB 00:00 2024-04-23T18:07:49.191 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 169 kB/s | 18 kB 00:00 2024-04-23T18:07:49.300 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-23T18:07:49.417 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 357 kB/s | 80 kB 00:00 2024-04-23T18:07:49.509 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-23T18:07:49.614 INFO:teuthology.orchestra.run.smithi173.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 4.7 MB 00:01 2024-04-23T18:07:49.951 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 12 MB/s | 49 MB 00:04 2024-04-23T18:07:49.976 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 542 kB/s | 253 kB 00:00 2024-04-23T18:07:50.010 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-23T18:07:50.052 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-23T18:07:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-23T18:07:50.111 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-23T18:07:50.269 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-04-23T18:07:50.312 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 952 kB/s | 215 kB 00:00 2024-04-23T18:07:50.362 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-23T18:07:50.404 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-04-23T18:07:50.437 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-04-23T18:07:50.462 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 709 kB/s | 107 kB 00:00 2024-04-23T18:07:50.488 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-04-23T18:07:50.513 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-23T18:07:50.538 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-23T18:07:50.563 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-23T18:07:50.589 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-23T18:07:50.589 INFO:teuthology.orchestra.run.smithi173.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.2 MB/s | 21 MB 00:05 2024-04-23T18:07:50.614 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-23T18:07:50.656 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-23T18:07:50.672 INFO:teuthology.orchestra.run.smithi173.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 303 kB/s | 25 kB 00:00 2024-04-23T18:07:50.698 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-23T18:07:50.732 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.8 MB/s | 582 kB 00:00 2024-04-23T18:07:50.757 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-04-23T18:07:50.782 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-23T18:07:50.808 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-23T18:07:50.824 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 846 kB/s | 35 kB 00:00 2024-04-23T18:07:50.883 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-04-23T18:07:50.983 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-23T18:07:51.017 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-23T18:07:51.031 INFO:teuthology.orchestra.run.smithi173.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 7.5 MB/s | 11 MB 00:01 2024-04-23T18:07:51.084 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-23T18:07:51.109 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-23T18:07:51.139 INFO:teuthology.orchestra.run.smithi173.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 225 kB 00:00 2024-04-23T18:07:51.143 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-23T18:07:51.185 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-23T18:07:51.202 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-23T18:07:51.231 INFO:teuthology.orchestra.run.smithi173.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 296 kB/s | 27 kB 00:00 2024-04-23T18:07:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T18:07:51.327 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-23T18:07:51.331 INFO:teuthology.orchestra.run.smithi173.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 317 kB/s | 32 kB 00:00 2024-04-23T18:07:51.353 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-23T18:07:51.377 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-23T18:07:51.403 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-23T18:07:51.429 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-23T18:07:51.454 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-23T18:07:51.515 INFO:teuthology.orchestra.run.smithi173.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.0 MB/s | 747 kB 00:00 2024-04-23T18:07:51.537 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-23T18:07:51.563 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-23T18:07:51.587 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-23T18:07:51.604 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-23T18:07:51.624 INFO:teuthology.orchestra.run.smithi173.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-23T18:07:51.630 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-23T18:07:51.655 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-23T18:07:51.664 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.2 MB 00:00 2024-04-23T18:07:51.680 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-23T18:07:51.706 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-23T18:07:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T18:07:51.732 INFO:teuthology.orchestra.run.smithi173.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.1 MB/s | 126 kB 00:00 2024-04-23T18:07:51.756 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T18:07:51.832 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-23T18:07:51.857 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-23T18:07:51.858 INFO:teuthology.orchestra.run.smithi173.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.9 MB/s | 500 kB 00:00 2024-04-23T18:07:51.882 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-04-23T18:07:51.907 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-23T18:07:51.933 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-23T18:07:51.958 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-23T18:07:52.000 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-23T18:07:52.025 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-04-23T18:07:52.051 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T18:07:52.084 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-23T18:07:52.109 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-23T18:07:52.335 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-04-23T18:07:52.609 INFO:teuthology.orchestra.run.smithi173.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.1 MB/s | 5.4 MB 00:00 2024-04-23T18:07:52.643 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-23T18:07:52.701 INFO:teuthology.orchestra.run.smithi173.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 494 kB/s | 45 kB 00:00 2024-04-23T18:07:52.718 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.1 MB/s | 309 kB 00:00 2024-04-23T18:07:52.777 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-04-23T18:07:52.801 INFO:teuthology.orchestra.run.smithi173.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-23T18:07:52.836 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-04-23T18:07:52.901 INFO:teuthology.orchestra.run.smithi173.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.6 MB/s | 161 kB 00:00 2024-04-23T18:07:52.993 INFO:teuthology.orchestra.run.smithi173.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-23T18:07:53.064 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-23T18:07:53.086 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 104 kB/s | 26 kB 00:00 2024-04-23T18:07:53.094 INFO:teuthology.orchestra.run.smithi173.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-23T18:07:53.120 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 868 kB/s | 49 kB 00:00 2024-04-23T18:07:53.177 INFO:teuthology.orchestra.run.smithi173.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-23T18:07:53.179 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-23T18:07:53.204 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-04-23T18:07:53.246 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-23T18:07:53.269 INFO:teuthology.orchestra.run.smithi173.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 932 kB/s | 85 kB 00:00 2024-04-23T18:07:53.304 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 746 kB/s | 43 kB 00:00 2024-04-23T18:07:53.363 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-04-23T18:07:53.397 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-23T18:07:53.422 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-23T18:07:53.456 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-23T18:07:53.506 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-23T18:07:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-23T18:07:53.565 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 599 kB/s | 35 kB 00:00 2024-04-23T18:07:53.591 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-23T18:07:53.649 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-23T18:07:53.709 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-04-23T18:07:53.711 INFO:teuthology.orchestra.run.smithi173.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.0 MB/s | 3.1 MB 00:00 2024-04-23T18:07:53.758 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-23T18:07:53.783 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-23T18:07:53.811 INFO:teuthology.orchestra.run.smithi173.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-23T18:07:53.817 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-23T18:07:53.842 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-23T18:07:53.876 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-23T18:07:53.903 INFO:teuthology.orchestra.run.smithi173.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-23T18:07:53.918 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-23T18:07:53.943 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-04-23T18:07:53.995 INFO:teuthology.orchestra.run.smithi173.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-23T18:07:53.996 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-23T18:07:54.027 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-04-23T18:07:54.059 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-23T18:07:54.060 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): protobuf-compiler-3.14.0-13.el9.x86_ 443 kB/s | 863 kB 00:01 2024-04-23T18:07:54.086 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 286 kB/s | 16 kB 00:00 2024-04-23T18:07:54.111 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-23T18:07:54.144 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-23T18:07:54.170 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-23T18:07:54.203 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-23T18:07:54.228 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-23T18:07:54.262 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-04-23T18:07:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 412 kB/s | 90 kB 00:00 2024-04-23T18:07:54.304 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-23T18:07:54.330 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-23T18:07:54.372 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-23T18:07:54.397 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-websocket-client-1.2.3-2.el9 763 kB/s | 90 kB 00:00 2024-04-23T18:07:54.414 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-23T18:07:54.429 INFO:teuthology.orchestra.run.smithi173.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 8.2 MB/s | 3.5 MB 00:00 2024-04-23T18:07:54.439 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-23T18:07:54.523 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-23T18:07:54.989 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 1.3 MB 00:00 2024-04-23T18:07:55.232 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.0 MB/s | 3.1 MB 00:00 2024-04-23T18:07:55.250 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-23T18:07:55.282 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.9 MB/s | 3.4 MB 00:00 2024-04-23T18:07:55.529 INFO:teuthology.orchestra.run.smithi173.stdout:(31/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 17 MB 00:06 2024-04-23T18:07:55.548 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 2.4 MB 00:00 2024-04-23T18:07:55.552 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:07:55.553 INFO:teuthology.orchestra.run.smithi037.stdout:Total 17 MB/s | 222 MB 00:12 2024-04-23T18:07:55.629 INFO:teuthology.orchestra.run.smithi173.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-04-23T18:07:55.654 INFO:teuthology.orchestra.run.smithi173.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.9 MB/s | 249 kB 00:00 2024-04-23T18:07:55.688 INFO:teuthology.orchestra.run.smithi173.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 845 kB/s | 49 kB 00:00 2024-04-23T18:07:55.713 INFO:teuthology.orchestra.run.smithi173.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 280 kB/s | 16 kB 00:00 2024-04-23T18:07:55.763 INFO:teuthology.orchestra.run.smithi173.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.6 MB/s | 273 kB 00:00 2024-04-23T18:07:56.248 INFO:teuthology.orchestra.run.smithi173.stdout:(37/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.8 MB/s | 49 MB 00:05 2024-04-23T18:07:56.272 INFO:teuthology.orchestra.run.smithi173.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 36 kB/s | 18 kB 00:00 2024-04-23T18:07:56.306 INFO:teuthology.orchestra.run.smithi173.stdout:(39/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.3 MB/s | 762 kB 00:00 2024-04-23T18:07:56.692 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-23T18:07:56.852 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-23T18:07:56.852 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-23T18:07:57.006 INFO:teuthology.orchestra.run.smithi173.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 106 kB/s | 80 kB 00:00 2024-04-23T18:07:57.173 INFO:teuthology.orchestra.run.smithi173.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 225 kB/s | 195 kB 00:00 2024-04-23T18:07:57.876 INFO:teuthology.orchestra.run.smithi173.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 501 kB/s | 803 kB 00:01 2024-04-23T18:07:58.001 INFO:teuthology.orchestra.run.smithi173.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 255 kB/s | 253 kB 00:00 2024-04-23T18:07:58.618 INFO:teuthology.orchestra.run.smithi173.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 144 kB/s | 106 kB 00:00 2024-04-23T18:07:58.728 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-23T18:07:58.729 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-23T18:07:58.776 INFO:teuthology.orchestra.run.smithi173.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 174 kB/s | 135 kB 00:00 2024-04-23T18:07:59.285 INFO:teuthology.orchestra.run.smithi173.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 595 kB/s | 1.2 MB 00:02 2024-04-23T18:07:59.410 INFO:teuthology.orchestra.run.smithi173.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 158 kB/s | 125 kB 00:00 2024-04-23T18:07:59.469 INFO:teuthology.orchestra.run.smithi173.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-23T18:07:59.511 INFO:teuthology.orchestra.run.smithi173.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-23T18:07:59.553 INFO:teuthology.orchestra.run.smithi173.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 516 kB/s | 21 kB 00:00 2024-04-23T18:07:59.595 INFO:teuthology.orchestra.run.smithi173.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-04-23T18:07:59.628 INFO:teuthology.orchestra.run.smithi173.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-23T18:07:59.654 INFO:teuthology.orchestra.run.smithi173.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-23T18:07:59.679 INFO:teuthology.orchestra.run.smithi173.stdout:(54/152): python3-urllib3-1.26.5-5.el9.noarch.r 238 kB/s | 215 kB 00:00 2024-04-23T18:07:59.704 INFO:teuthology.orchestra.run.smithi173.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-23T18:07:59.746 INFO:teuthology.orchestra.run.smithi173.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-23T18:07:59.771 INFO:teuthology.orchestra.run.smithi173.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-23T18:07:59.830 INFO:teuthology.orchestra.run.smithi173.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-23T18:07:59.857 INFO:teuthology.orchestra.run.smithi173.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-04-23T18:07:59.899 INFO:teuthology.orchestra.run.smithi173.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.3 MB/s | 582 kB 00:00 2024-04-23T18:07:59.922 INFO:teuthology.orchestra.run.smithi173.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-04-23T18:07:59.956 INFO:teuthology.orchestra.run.smithi173.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-23T18:07:59.981 INFO:teuthology.orchestra.run.smithi173.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 579 kB/s | 47 kB 00:00 2024-04-23T18:08:00.006 INFO:teuthology.orchestra.run.smithi173.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-23T18:08:00.031 INFO:teuthology.orchestra.run.smithi173.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-23T18:08:00.057 INFO:teuthology.orchestra.run.smithi173.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-23T18:08:00.149 INFO:teuthology.orchestra.run.smithi173.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-23T18:08:00.216 INFO:teuthology.orchestra.run.smithi173.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-23T18:08:00.249 INFO:teuthology.orchestra.run.smithi173.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-23T18:08:00.283 INFO:teuthology.orchestra.run.smithi173.stdout:(70/152): postgresql-13.14-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-23T18:08:00.300 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-23T18:08:00.308 INFO:teuthology.orchestra.run.smithi173.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-23T18:08:00.351 INFO:teuthology.orchestra.run.smithi173.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-23T18:08:00.376 INFO:teuthology.orchestra.run.smithi173.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-23T18:08:00.418 INFO:teuthology.orchestra.run.smithi173.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T18:08:00.493 INFO:teuthology.orchestra.run.smithi173.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-23T18:08:00.526 INFO:teuthology.orchestra.run.smithi173.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-23T18:08:00.543 INFO:teuthology.orchestra.run.smithi173.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-23T18:08:00.560 INFO:teuthology.orchestra.run.smithi173.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 570 kB/s | 19 kB 00:00 2024-04-23T18:08:00.586 INFO:teuthology.orchestra.run.smithi173.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-23T18:08:00.628 INFO:teuthology.orchestra.run.smithi173.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-23T18:08:00.703 INFO:teuthology.orchestra.run.smithi173.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-23T18:08:00.737 INFO:teuthology.orchestra.run.smithi173.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-23T18:08:00.762 INFO:teuthology.orchestra.run.smithi173.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-23T18:08:00.787 INFO:teuthology.orchestra.run.smithi173.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-23T18:08:00.812 INFO:teuthology.orchestra.run.smithi173.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-23T18:08:00.838 INFO:teuthology.orchestra.run.smithi173.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-23T18:08:00.863 INFO:teuthology.orchestra.run.smithi173.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-23T18:08:00.888 INFO:teuthology.orchestra.run.smithi173.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-23T18:08:00.914 INFO:teuthology.orchestra.run.smithi173.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T18:08:00.939 INFO:teuthology.orchestra.run.smithi173.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T18:08:01.023 INFO:teuthology.orchestra.run.smithi173.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-23T18:08:01.047 INFO:teuthology.orchestra.run.smithi173.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-23T18:08:01.073 INFO:teuthology.orchestra.run.smithi173.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-23T18:08:01.098 INFO:teuthology.orchestra.run.smithi173.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-23T18:08:01.123 INFO:teuthology.orchestra.run.smithi173.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-23T18:08:01.149 INFO:teuthology.orchestra.run.smithi173.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-23T18:08:01.182 INFO:teuthology.orchestra.run.smithi173.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-23T18:08:01.208 INFO:teuthology.orchestra.run.smithi173.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-04-23T18:08:01.233 INFO:teuthology.orchestra.run.smithi173.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T18:08:01.258 INFO:teuthology.orchestra.run.smithi173.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-23T18:08:01.284 INFO:teuthology.orchestra.run.smithi173.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-23T18:08:01.367 INFO:teuthology.orchestra.run.smithi173.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-23T18:08:01.560 INFO:teuthology.orchestra.run.smithi173.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-23T18:08:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-04-23T18:08:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-23T18:08:01.827 INFO:teuthology.orchestra.run.smithi173.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-04-23T18:08:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-23T18:08:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-23T18:08:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 135 kB/s | 26 kB 00:00 2024-04-23T18:08:02.044 INFO:teuthology.orchestra.run.smithi173.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 310 kB/s | 49 kB 00:00 2024-04-23T18:08:02.070 INFO:teuthology.orchestra.run.smithi173.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-04-23T18:08:02.095 INFO:teuthology.orchestra.run.smithi173.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-23T18:08:02.129 INFO:teuthology.orchestra.run.smithi173.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-23T18:08:02.162 INFO:teuthology.orchestra.run.smithi173.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-23T18:08:02.187 INFO:teuthology.orchestra.run.smithi173.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 481 kB/s | 43 kB 00:00 2024-04-23T18:08:02.213 INFO:teuthology.orchestra.run.smithi173.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 646 kB/s | 32 kB 00:00 2024-04-23T18:08:02.238 INFO:teuthology.orchestra.run.smithi173.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-04-23T18:08:02.263 INFO:teuthology.orchestra.run.smithi173.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-23T18:08:02.288 INFO:teuthology.orchestra.run.smithi173.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-23T18:08:02.313 INFO:teuthology.orchestra.run.smithi173.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-23T18:08:02.330 INFO:teuthology.orchestra.run.smithi173.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 538 kB/s | 35 kB 00:00 2024-04-23T18:08:02.355 INFO:teuthology.orchestra.run.smithi173.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-23T18:08:02.401 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:08:02.401 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:08:02.412 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:08:02.413 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:08:02.414 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:08:02.414 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:08:02.416 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:08:02.418 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:08:02.419 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:08:02.422 INFO:teuthology.orchestra.run.smithi173.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-23T18:08:02.481 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-04-23T18:08:02.501 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-23T18:08:02.501 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-04-23T18:08:02.501 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-23T18:08:02.501 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-04-23T18:08:02.501 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:08:02.501 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:08:02.502 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:08:02.503 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:08:02.504 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:08:02.505 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:08:02.506 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:08:02.507 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:08:02.508 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:08:02.509 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:08:02.510 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:08:02.511 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:08:02.512 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:08:02.513 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-04-23T18:08:02.514 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-23T18:08:02.515 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-04-23T18:08:02.515 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 4 Packages 2024-04-23T18:08:02.515 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:08:02.516 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 222 M 2024-04-23T18:08:02.516 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-04-23T18:08:02.522 INFO:teuthology.orchestra.run.smithi173.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 754 kB/s | 144 kB 00:00 2024-04-23T18:08:02.631 INFO:teuthology.orchestra.run.smithi173.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 39 kB/s | 11 kB 00:00 2024-04-23T18:08:02.657 INFO:teuthology.orchestra.run.smithi173.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 76 kB/s | 18 kB 00:00 2024-04-23T18:08:02.682 INFO:teuthology.orchestra.run.smithi173.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-04-23T18:08:02.707 INFO:teuthology.orchestra.run.smithi173.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-23T18:08:02.732 INFO:teuthology.orchestra.run.smithi173.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-23T18:08:02.774 INFO:teuthology.orchestra.run.smithi173.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-23T18:08:02.799 INFO:teuthology.orchestra.run.smithi173.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-04-23T18:08:02.825 INFO:teuthology.orchestra.run.smithi173.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 870 kB/s | 79 kB 00:00 2024-04-23T18:08:02.850 INFO:teuthology.orchestra.run.smithi173.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-23T18:08:02.884 INFO:teuthology.orchestra.run.smithi173.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-23T18:08:02.909 INFO:teuthology.orchestra.run.smithi173.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-23T18:08:02.934 INFO:teuthology.orchestra.run.smithi173.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-23T18:08:02.960 INFO:teuthology.orchestra.run.smithi173.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 567 kB/s | 28 kB 00:00 2024-04-23T18:08:02.993 INFO:teuthology.orchestra.run.smithi173.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 370 kB/s | 31 kB 00:00 2024-04-23T18:08:03.018 INFO:teuthology.orchestra.run.smithi173.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-23T18:08:03.043 INFO:teuthology.orchestra.run.smithi173.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 721 kB/s | 36 kB 00:00 2024-04-23T18:08:03.069 INFO:teuthology.orchestra.run.smithi173.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 789 kB/s | 59 kB 00:00 2024-04-23T18:08:03.094 INFO:teuthology.orchestra.run.smithi173.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-23T18:08:03.119 INFO:teuthology.orchestra.run.smithi173.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-23T18:08:03.144 INFO:teuthology.orchestra.run.smithi173.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-23T18:08:03.169 INFO:teuthology.orchestra.run.smithi173.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-23T18:08:03.195 INFO:teuthology.orchestra.run.smithi173.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-04-23T18:08:03.220 INFO:teuthology.orchestra.run.smithi173.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-23T18:08:03.262 INFO:teuthology.orchestra.run.smithi173.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-23T18:08:03.485 INFO:teuthology.orchestra.run.smithi172.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-23T18:08:03.729 INFO:teuthology.orchestra.run.smithi173.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.4 MB/s | 3.4 MB 00:00 2024-04-23T18:08:03.787 INFO:teuthology.orchestra.run.smithi173.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-23T18:08:03.868 INFO:teuthology.orchestra.run.smithi172.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 890 kB 00:00 2024-04-23T18:08:03.944 INFO:teuthology.orchestra.run.smithi172.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-23T18:08:04.552 INFO:teuthology.orchestra.run.smithi172.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.9 MB/s | 2.4 MB 00:00 2024-04-23T18:08:04.694 INFO:teuthology.orchestra.run.smithi172.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-23T18:08:04.841 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-23T18:08:04.953 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:08:05.003 INFO:teuthology.orchestra.run.smithi172.stdout:(6/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 12 MB/s | 21 MB 00:01 2024-04-23T18:08:05.044 INFO:teuthology.orchestra.run.smithi172.stdout:(7/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.0 MB/s | 1.5 MB 00:00 2024-04-23T18:08:05.135 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:08:05.189 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:08:05.230 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:08:05.262 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:08:05.308 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:08:05.570 INFO:teuthology.orchestra.run.smithi172.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.4 MB/s | 4.7 MB 00:00 2024-04-23T18:08:05.651 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:08:05.653 INFO:teuthology.orchestra.run.smithi172.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 302 kB/s | 25 kB 00:00 2024-04-23T18:08:05.696 INFO:teuthology.orchestra.run.smithi173.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 568 kB/s | 1.3 MB 00:02 2024-04-23T18:08:05.755 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:08:05.880 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:08:05.906 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:08:05.999 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:08:06.062 INFO:teuthology.orchestra.run.smithi172.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 16 MB/s | 17 MB 00:01 2024-04-23T18:08:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:08:06.128 INFO:teuthology.orchestra.run.smithi172.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.3 MB/s | 225 kB 00:00 2024-04-23T18:08:06.187 INFO:teuthology.orchestra.run.smithi172.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 464 kB/s | 27 kB 00:00 2024-04-23T18:08:06.246 INFO:teuthology.orchestra.run.smithi172.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 543 kB/s | 32 kB 00:00 2024-04-23T18:08:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:08:06.341 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:08:06.346 INFO:teuthology.orchestra.run.smithi172.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 7.3 MB/s | 747 kB 00:00 2024-04-23T18:08:06.368 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:08:06.405 INFO:teuthology.orchestra.run.smithi172.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-23T18:08:06.474 INFO:teuthology.orchestra.run.smithi172.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.8 MB/s | 126 kB 00:00 2024-04-23T18:08:06.475 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:08:06.503 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:08:06.557 INFO:teuthology.orchestra.run.smithi172.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.9 MB/s | 500 kB 00:00 2024-04-23T18:08:06.566 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:08:06.587 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:08:06.608 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:08:06.796 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:08:06.829 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:08:06.941 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:08:07.037 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:08:07.083 INFO:teuthology.orchestra.run.smithi172.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 5.4 MB 00:00 2024-04-23T18:08:07.142 INFO:teuthology.orchestra.run.smithi172.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 776 kB/s | 45 kB 00:00 2024-04-23T18:08:07.178 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:08:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:08:07.284 INFO:teuthology.orchestra.run.smithi172.stdout:(20/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 4.7 MB/s | 11 MB 00:02 2024-04-23T18:08:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:08:07.296 INFO:teuthology.orchestra.run.smithi173.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 687 kB/s | 2.4 MB 00:03 2024-04-23T18:08:07.302 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:08:07.302 INFO:teuthology.orchestra.run.smithi173.stdout:Total 9.9 MB/s | 222 MB 00:22 2024-04-23T18:08:07.309 INFO:teuthology.orchestra.run.smithi172.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 808 kB/s | 135 kB 00:00 2024-04-23T18:08:07.329 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:08:07.368 INFO:teuthology.orchestra.run.smithi172.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 5.4 MB/s | 320 kB 00:00 2024-04-23T18:08:07.373 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:08:07.393 INFO:teuthology.orchestra.run.smithi172.stdout:(23/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-23T18:08:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:08:07.426 INFO:teuthology.orchestra.run.smithi172.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 5.0 MB/s | 298 kB 00:00 2024-04-23T18:08:07.458 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:08:07.469 INFO:teuthology.orchestra.run.smithi172.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-23T18:08:07.494 INFO:teuthology.orchestra.run.smithi172.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 85 kB 00:00 2024-04-23T18:08:07.531 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:08:07.552 INFO:teuthology.orchestra.run.smithi172.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 170 kB 00:00 2024-04-23T18:08:07.581 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:08:07.611 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:08:07.611 INFO:teuthology.orchestra.run.smithi172.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 587 kB/s | 34 kB 00:00 2024-04-23T18:08:07.655 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:08:07.670 INFO:teuthology.orchestra.run.smithi172.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.4 MB/s | 142 kB 00:00 2024-04-23T18:08:07.733 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:08:07.765 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:08:07.808 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:08:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:08:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:08:07.977 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:08:08.037 INFO:teuthology.orchestra.run.smithi172.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 9.7 MB/s | 3.5 MB 00:00 2024-04-23T18:08:08.058 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:08:08.205 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:08:08.256 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:08:08.270 INFO:teuthology.orchestra.run.smithi172.stdout:(31/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.9 MB/s | 3.1 MB 00:00 2024-04-23T18:08:08.361 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-23T18:08:08.387 INFO:teuthology.orchestra.run.smithi172.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.1 MB/s | 249 kB 00:00 2024-04-23T18:08:08.496 INFO:teuthology.orchestra.run.smithi172.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 455 kB/s | 49 kB 00:00 2024-04-23T18:08:08.505 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-23T18:08:08.505 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-23T18:08:08.604 INFO:teuthology.orchestra.run.smithi172.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-23T18:08:08.696 INFO:teuthology.orchestra.run.smithi172.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-23T18:08:08.722 INFO:teuthology.orchestra.run.smithi172.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 273 kB 00:00 2024-04-23T18:08:08.797 INFO:teuthology.orchestra.run.smithi172.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 7.4 MB/s | 762 kB 00:00 2024-04-23T18:08:08.822 INFO:teuthology.orchestra.run.smithi172.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 182 kB/s | 18 kB 00:00 2024-04-23T18:08:08.826 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:08:08.912 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:08:08.945 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:08:08.988 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:08:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:08:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:08:09.222 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:08:09.273 INFO:teuthology.orchestra.run.smithi172.stdout:(39/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 14 MB/s | 49 MB 00:03 2024-04-23T18:08:09.279 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:08:09.298 INFO:teuthology.orchestra.run.smithi172.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 160 kB/s | 80 kB 00:00 2024-04-23T18:08:09.336 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:08:09.398 INFO:teuthology.orchestra.run.smithi172.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-23T18:08:09.410 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:08:09.432 INFO:teuthology.orchestra.run.smithi172.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-23T18:08:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:08:09.457 INFO:teuthology.orchestra.run.smithi172.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-23T18:08:09.483 INFO:teuthology.orchestra.run.smithi172.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T18:08:09.491 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:08:09.508 INFO:teuthology.orchestra.run.smithi172.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-23T18:08:09.533 INFO:teuthology.orchestra.run.smithi172.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-23T18:08:09.540 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:08:09.558 INFO:teuthology.orchestra.run.smithi172.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-23T18:08:09.574 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:08:09.592 INFO:teuthology.orchestra.run.smithi172.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-04-23T18:08:09.618 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:08:09.660 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:08:09.806 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:08:09.857 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:08:09.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:08:09.929 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:08:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:08:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:08:10.101 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:08:10.134 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:08:10.159 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:08:10.231 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:08:10.275 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-23T18:08:10.276 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-23T18:08:10.326 INFO:teuthology.orchestra.run.smithi172.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-04-23T18:08:10.369 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:08:10.587 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:08:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:08:10.917 INFO:teuthology.orchestra.run.smithi172.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-04-23T18:08:10.930 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:08:11.013 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:08:11.026 INFO:teuthology.orchestra.run.smithi172.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 462 kB/s | 689 kB 00:01 2024-04-23T18:08:11.061 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:08:11.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:08:11.141 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:08:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:08:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:08:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:08:11.318 INFO:teuthology.orchestra.run.smithi172.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 536 kB/s | 943 kB 00:01 2024-04-23T18:08:11.346 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:08:11.398 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:08:11.436 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:08:11.485 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:08:11.597 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:08:11.626 INFO:teuthology.orchestra.run.smithi172.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-04-23T18:08:11.646 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:08:11.926 INFO:teuthology.orchestra.run.smithi172.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 311 kB/s | 313 kB 00:01 2024-04-23T18:08:12.143 INFO:teuthology.orchestra.run.smithi172.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-04-23T18:08:12.437 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:08:12.460 INFO:teuthology.orchestra.run.smithi172.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-04-23T18:08:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:08:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:08:12.625 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:08:12.677 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:08:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:08:12.812 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:08:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:08:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:08:13.052 INFO:teuthology.orchestra.run.smithi172.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 176 kB/s | 160 kB 00:00 2024-04-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi172.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 63 kB/s | 45 kB 00:00 2024-04-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi172.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 431 kB/s | 582 kB 00:01 2024-04-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi172.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 441 kB/s | 662 kB 00:01 2024-04-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi172.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 43 kB/s | 25 kB 00:00 2024-04-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi172.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-04-23T18:08:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:08:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:08:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:08:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:08:15.703 INFO:teuthology.orchestra.run.smithi172.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 698 kB/s | 3.0 MB 00:04 2024-04-23T18:08:15.728 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:08:15.779 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:08:15.870 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:08:15.900 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:08:15.937 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:08:16.070 INFO:teuthology.orchestra.run.smithi172.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 213 kB/s | 188 kB 00:00 2024-04-23T18:08:16.171 INFO:teuthology.orchestra.run.smithi172.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 278 kB/s | 292 kB 00:01 2024-04-23T18:08:16.354 INFO:teuthology.orchestra.run.smithi172.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 54 kB/s | 35 kB 00:00 2024-04-23T18:08:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:08:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:08:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:08:16.555 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-23T18:08:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:08:16.633 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:08:16.671 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:08:16.725 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:08:16.758 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:08:16.852 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:08:17.182 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:08:17.295 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:08:17.415 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:08:17.444 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:08:17.535 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:08:17.574 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:08:17.720 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:08:17.770 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:08:17.798 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:08:17.902 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:08:17.924 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:08:17.987 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:08:18.009 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:08:18.033 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:08:18.226 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:08:18.258 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:08:18.366 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:08:18.460 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:08:18.499 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:08:18.534 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:08:18.597 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:08:18.633 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:08:18.677 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:08:18.709 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:08:18.746 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:08:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:08:18.885 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:08:18.918 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:08:18.959 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:08:18.969 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:08:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:08:19.029 INFO:teuthology.orchestra.run.smithi172.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 608 kB/s | 1.6 MB 00:02 2024-04-23T18:08:19.036 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:08:19.070 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:08:19.113 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:08:19.142 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:08:19.212 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:08:19.273 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:08:19.364 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:08:19.496 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:08:19.553 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:08:19.578 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:08:19.617 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:08:19.720 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:08:19.720 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:08:19.720 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:08:19.720 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:19.751 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:08:20.069 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:08:20.146 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:08:20.175 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:08:20.210 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:08:20.248 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:08:20.289 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:08:20.454 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:08:20.517 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:08:20.573 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:08:20.653 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:08:20.685 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:08:20.729 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:08:20.776 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:08:20.812 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:08:20.837 INFO:teuthology.orchestra.run.smithi172.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 504 kB/s | 911 kB 00:01 2024-04-23T18:08:20.855 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:08:20.898 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:08:21.031 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:08:21.078 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:08:21.108 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:08:21.158 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:08:21.252 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:08:21.291 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:08:21.348 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:08:21.374 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:08:21.399 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:08:21.472 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi172.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 176 kB/s | 133 kB 00:00 2024-04-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:08:21.659 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:08:21.701 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:08:21.749 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:08:21.780 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:08:21.810 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:08:21.845 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:08:21.878 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:08:21.919 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:08:21.954 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:08:22.041 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:08:22.085 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:08:22.125 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:08:22.172 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:08:22.269 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:08:22.318 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:08:22.346 INFO:teuthology.orchestra.run.smithi172.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 734 kB/s | 4.4 MB 00:06 2024-04-23T18:08:22.588 INFO:teuthology.orchestra.run.smithi172.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 745 kB/s | 4.7 MB 00:06 2024-04-23T18:08:23.028 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:08:23.082 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:08:23.115 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:08:23.221 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:08:23.273 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:08:23.313 INFO:teuthology.orchestra.run.smithi172.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 125 kB/s | 87 kB 00:00 2024-04-23T18:08:23.360 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:08:23.401 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:08:23.578 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:08:23.607 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:08:23.684 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:08:24.247 INFO:teuthology.orchestra.run.smithi172.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 544 kB/s | 1.0 MB 00:01 2024-04-23T18:08:24.315 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:08:24.420 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:08:25.164 INFO:teuthology.orchestra.run.smithi172.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 266 kB/s | 244 kB 00:00 2024-04-23T18:08:25.635 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:08:25.710 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:08:25.738 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:08:25.780 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:08:26.039 INFO:teuthology.orchestra.run.smithi172.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 283 kB/s | 248 kB 00:00 2024-04-23T18:08:26.092 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:08:26.160 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:08:26.250 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:08:26.281 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:08:26.317 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:08:26.614 INFO:teuthology.orchestra.run.smithi172.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-04-23T18:08:26.796 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:08:26.829 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:08:26.901 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:08:26.923 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:08:27.156 INFO:teuthology.orchestra.run.smithi172.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-04-23T18:08:27.990 INFO:teuthology.orchestra.run.smithi172.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 207 kB/s | 172 kB 00:00 2024-04-23T18:08:28.598 INFO:teuthology.orchestra.run.smithi172.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-04-23T18:08:29.131 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:08:29.336 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:08:29.465 INFO:teuthology.orchestra.run.smithi172.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 752 kB/s | 5.8 MB 00:07 2024-04-23T18:08:29.881 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:08:29.914 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:08:30.006 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:08:30.006 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:08:30.006 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:08:30.006 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:30.040 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:08:30.399 INFO:teuthology.orchestra.run.smithi172.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 269 kB/s | 250 kB 00:00 2024-04-23T18:08:31.157 INFO:teuthology.orchestra.run.smithi172.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 778 kB/s | 6.0 MB 00:07 2024-04-23T18:08:31.300 INFO:teuthology.orchestra.run.smithi172.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 247 kB/s | 222 kB 00:00 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-23T18:08:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:08:31.914 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:08:31.914 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:31.914 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:08:31.914 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:31.966 INFO:teuthology.orchestra.run.smithi172.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 95 kB/s | 77 kB 00:00 2024-04-23T18:08:31.991 INFO:teuthology.orchestra.run.smithi172.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 61 kB/s | 42 kB 00:00 2024-04-23T18:08:32.783 INFO:teuthology.orchestra.run.smithi172.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 201 kB/s | 159 kB 00:00 2024-04-23T18:08:32.900 INFO:teuthology.orchestra.run.smithi172.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 288 kB/s | 269 kB 00:00 2024-04-23T18:08:33.450 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:08:33.477 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:08:33.478 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:33.478 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:08:33.478 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:08:33.478 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:08:33.478 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi172.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-04-23T18:08:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:08:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:08:33.733 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:33.733 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:08:33.733 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:08:33.733 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:08:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:33.775 INFO:teuthology.orchestra.run.smithi172.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 281 kB/s | 279 kB 00:00 2024-04-23T18:08:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:08:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:34.218 INFO:teuthology.orchestra.run.smithi172.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 81 kB/s | 54 kB 00:00 2024-04-23T18:08:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:08:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:08:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:08:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:08:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:08:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:08:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:08:34.867 INFO:teuthology.orchestra.run.smithi172.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-04-23T18:08:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:08:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:08:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:08:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:08:35.493 INFO:teuthology.orchestra.run.smithi172.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-04-23T18:08:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:08:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:08:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:08:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:08:35.676 INFO:teuthology.orchestra.run.smithi172.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 737 kB/s | 5.1 MB 00:07 2024-04-23T18:08:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:08:36.293 INFO:teuthology.orchestra.run.smithi172.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 58 kB/s | 36 kB 00:00 2024-04-23T18:08:36.393 INFO:teuthology.orchestra.run.smithi172.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 245 kB/s | 221 kB 00:00 2024-04-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:08:36.827 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:08:36.968 INFO:teuthology.orchestra.run.smithi172.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-04-23T18:08:37.377 INFO:teuthology.orchestra.run.smithi172.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 310 kB/s | 305 kB 00:00 2024-04-23T18:08:37.652 INFO:teuthology.orchestra.run.smithi172.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 85 kB/s | 58 kB 00:00 2024-04-23T18:08:37.845 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:08:37.877 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:08:37.988 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:08:38.086 INFO:teuthology.orchestra.run.smithi172.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 132 kB/s | 94 kB 00:00 2024-04-23T18:08:38.157 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:08:38.188 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:08:38.212 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:08:38.212 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:38.212 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:08:38.212 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:08:38.213 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:08:38.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:08:38.319 INFO:teuthology.orchestra.run.smithi172.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 95 kB/s | 64 kB 00:00 2024-04-23T18:08:38.377 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:08:38.412 INFO:teuthology.orchestra.run.smithi172.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 345 kB/s | 22 kB 00:00 2024-04-23T18:08:38.429 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:08:38.545 INFO:teuthology.orchestra.run.smithi172.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-23T18:08:38.729 INFO:teuthology.orchestra.run.smithi172.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-04-23T18:08:39.137 INFO:teuthology.orchestra.run.smithi172.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 926 kB/s | 548 kB 00:00 2024-04-23T18:08:39.254 INFO:teuthology.orchestra.run.smithi172.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-04-23T18:08:39.279 INFO:teuthology.orchestra.run.smithi172.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-04-23T18:08:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:08:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:08:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:08:39.466 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:39.466 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:08:39.466 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:08:39.466 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:08:39.466 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:39.508 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:08:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:08:39.532 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:39.532 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:08:39.532 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:39.538 INFO:teuthology.orchestra.run.smithi172.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-23T18:08:39.584 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:08:39.608 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:08:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:08:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:08:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:08:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:39.638 INFO:teuthology.orchestra.run.smithi172.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 487 kB/s | 49 kB 00:00 2024-04-23T18:08:39.881 INFO:teuthology.orchestra.run.smithi172.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-23T18:08:39.926 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:08:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:08:39.950 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:39.950 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:08:39.950 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:08:39.950 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:08:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:39.964 INFO:teuthology.orchestra.run.smithi172.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 207 kB/s | 67 kB 00:00 2024-04-23T18:08:39.998 INFO:teuthology.orchestra.run.smithi172.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-23T18:08:40.498 INFO:teuthology.orchestra.run.smithi172.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.5 MB/s | 838 kB 00:00 2024-04-23T18:08:40.523 INFO:teuthology.orchestra.run.smithi172.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 1.0 MB/s | 548 kB 00:00 2024-04-23T18:08:40.590 INFO:teuthology.orchestra.run.smithi172.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-04-23T18:08:40.899 INFO:teuthology.orchestra.run.smithi172.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 86 kB/s | 32 kB 00:00 2024-04-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi172.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 37 kB/s | 15 kB 00:00 2024-04-23T18:08:41.016 INFO:teuthology.orchestra.run.smithi172.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-23T18:08:41.358 INFO:teuthology.orchestra.run.smithi172.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 977 kB/s | 358 kB 00:00 2024-04-23T18:08:41.458 INFO:teuthology.orchestra.run.smithi172.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 686 kB/s | 303 kB 00:00 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-04-23T18:08:41.519 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:41.583 INFO:teuthology.orchestra.run.smithi172.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 157 kB/s | 35 kB 00:00 2024-04-23T18:08:41.701 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:08:41.717 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:08:41.717 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:41.718 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:08:41.718 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:42.150 INFO:teuthology.orchestra.run.smithi172.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 297 kB/s | 206 kB 00:00 2024-04-23T18:08:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-23T18:08:42.359 INFO:teuthology.orchestra.run.smithi172.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-23T18:08:42.451 INFO:teuthology.orchestra.run.smithi172.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-23T18:08:42.476 INFO:teuthology.orchestra.run.smithi172.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-23T18:08:42.552 INFO:teuthology.orchestra.run.smithi172.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-23T18:08:42.569 INFO:teuthology.orchestra.run.smithi172.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-23T18:08:42.644 INFO:teuthology.orchestra.run.smithi172.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-23T18:08:42.744 INFO:teuthology.orchestra.run.smithi172.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-23T18:08:42.769 INFO:teuthology.orchestra.run.smithi172.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-23T18:08:42.837 INFO:teuthology.orchestra.run.smithi172.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2024-04-23T18:08:43.204 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:08:43.229 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:08:43.230 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:43.230 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:08:43.230 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:08:43.230 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:08:43.230 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:43.237 INFO:teuthology.orchestra.run.smithi172.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 124 kB/s | 58 kB 00:00 2024-04-23T18:08:43.262 INFO:teuthology.orchestra.run.smithi172.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 641 kB/s | 272 kB 00:00 2024-04-23T18:08:43.321 INFO:teuthology.orchestra.run.smithi172.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-23T18:08:43.396 INFO:teuthology.orchestra.run.smithi172.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 672 kB/s | 90 kB 00:00 2024-04-23T18:08:43.454 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:08:43.471 INFO:teuthology.orchestra.run.smithi172.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 188 kB/s | 28 kB 00:00 2024-04-23T18:08:43.476 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:08:43.476 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:43.476 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:08:43.477 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:08:43.477 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:08:43.477 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:43.563 INFO:teuthology.orchestra.run.smithi172.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 184 kB/s | 31 kB 00:00 2024-04-23T18:08:43.597 INFO:teuthology.orchestra.run.smithi172.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-23T18:08:43.922 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:08:43.944 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:08:43.944 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:43.944 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:08:43.944 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:08:43.944 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:08:43.944 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:44.052 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:08:44.064 INFO:teuthology.orchestra.run.smithi172.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 118 kB/s | 59 kB 00:00 2024-04-23T18:08:44.089 INFO:teuthology.orchestra.run.smithi172.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 73 kB/s | 36 kB 00:00 2024-04-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:08:44.156 INFO:teuthology.orchestra.run.smithi172.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 556 kB/s | 51 kB 00:00 2024-04-23T18:08:44.172 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:08:44.190 INFO:teuthology.orchestra.run.smithi172.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-23T18:08:44.359 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:08:44.390 INFO:teuthology.orchestra.run.smithi172.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 384 kB/s | 90 kB 00:00 2024-04-23T18:08:44.414 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:08:44.423 INFO:teuthology.orchestra.run.smithi172.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.8 MB/s | 427 kB 00:00 2024-04-23T18:08:44.482 INFO:teuthology.orchestra.run.smithi172.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-23T18:08:44.509 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:08:44.532 INFO:teuthology.orchestra.run.smithi172.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:08:44.632 INFO:teuthology.orchestra.run.smithi172.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-23T18:08:44.747 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:08:44.789 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:08:44.805 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:08:44.821 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:08:44.847 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:08:44.849 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:08:44.889 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:08:44.921 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:08:44.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:08:45.024 INFO:teuthology.orchestra.run.smithi172.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 8.0 MB/s | 3.1 MB 00:00 2024-04-23T18:08:45.154 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:08:45.175 INFO:teuthology.orchestra.run.smithi172.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-04-23T18:08:45.193 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:08:45.222 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:08:45.225 INFO:teuthology.orchestra.run.smithi172.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.9 MB/s | 3.4 MB 00:00 2024-04-23T18:08:45.284 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:08:45.292 INFO:teuthology.orchestra.run.smithi172.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-23T18:08:45.314 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:08:45.346 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:08:45.407 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:08:45.446 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:08:45.841 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:08:45.878 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:08:45.899 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:08:46.483 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:08:46.522 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:08:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:08:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:08:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:08:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:08:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:08:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:08:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:08:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:08:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:08:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:08:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:08:47.488 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:08:47.514 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:08:47.616 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:08:47.778 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:08:47.808 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:08:47.831 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:08:47.831 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:47.831 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:08:47.831 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:08:47.831 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:08:47.831 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:47.876 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:08:47.887 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:08:47.984 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:08:48.032 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:08:48.958 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:08:48.995 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:08:49.017 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:08:49.017 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:49.017 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:08:49.017 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:08:49.018 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:08:49.018 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:49.063 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:08:49.084 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:08:49.084 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:49.084 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:08:49.085 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:49.130 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:08:49.152 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:08:49.153 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:49.153 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:08:49.153 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:08:49.154 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:08:49.154 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:49.455 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:08:49.477 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:08:49.477 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:08:49.478 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:08:49.478 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:08:49.478 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:08:49.478 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:08:53.219 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:08:53.220 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:08:53.221 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:08:53.221 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:08:53.221 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:08:53.221 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:08:53.233 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:08:53.234 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:08:53.235 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:08:53.236 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:08:53.237 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:08:53.238 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:08:53.239 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:08:53.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:08:53.241 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:08:53.242 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:08:54.068 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:08:54.108 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:08:54.150 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:08:54.181 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:08:54.217 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:08:54.398 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:08:54.438 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:08:54.468 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:08:55.081 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:08:55.124 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:08:55.143 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:08:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:08:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:08:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:08:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:08:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:08:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:08:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:08:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:08:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:08:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:08:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:08:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:08:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:08:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:08:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:08:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:08:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:08:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:08:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:08:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:08:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:08:55.348 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-23T18:08:55.482 INFO:teuthology.orchestra.run.smithi172.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 767 kB/s | 16 MB 00:21 2024-04-23T18:08:55.485 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:08:55.486 INFO:teuthology.orchestra.run.smithi172.stdout:Total 4.2 MB/s | 222 MB 00:52 2024-04-23T18:08:55.756 DEBUG:teuthology.parallel:result is None 2024-04-23T18:08:55.973 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:08:56.062 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:08:56.102 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:08:56.192 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:08:56.232 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:08:56.233 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:08:56.316 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:08:56.341 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:08:56.341 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:08:56.435 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:08:56.506 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:08:56.550 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-04-23T18:08:56.695 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-04-23T18:08:56.695 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-04-23T18:08:58.464 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-04-23T18:08:58.464 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-04-23T18:09:02.574 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:09:02.574 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:09:02.574 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:09:02.574 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:09:02.575 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:09:02.576 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:09:02.577 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:09:02.578 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:09:02.579 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:09:02.580 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:09:02.581 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:09:02.582 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:09:02.583 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:09:02.584 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:09:02.585 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:09:02.586 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:09:04.384 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-04-23T18:09:04.469 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:09:04.515 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:09:04.569 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:09:04.599 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:09:04.599 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.600 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.601 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:09:04.602 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:09:04.603 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:09:04.604 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:09:04.605 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:09:04.606 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:09:04.607 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:09:04.608 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.609 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:09:04.610 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:09:04.611 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:09:04.612 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:09:04.612 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:09:04.612 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:09:04.612 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-23T18:09:04.637 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:09:04.679 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:09:04.998 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:09:05.021 DEBUG:teuthology.parallel:result is None 2024-04-23T18:09:05.104 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:09:05.226 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:09:05.256 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:09:05.347 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:09:05.377 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:09:05.523 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:09:05.573 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:09:05.602 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:09:05.704 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:09:05.728 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:09:05.774 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:09:05.795 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:09:05.819 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:09:05.987 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:09:06.020 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:09:06.127 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:09:06.214 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:09:06.253 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:09:06.287 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:09:06.353 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:09:06.395 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:09:06.439 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:09:06.471 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:09:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:09:06.568 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:09:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:09:06.652 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:09:06.697 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:09:06.782 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:09:06.815 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:09:06.850 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:09:06.879 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:09:06.949 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:09:06.994 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:09:07.068 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:09:07.218 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:09:07.271 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:09:07.800 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:09:07.883 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:09:07.912 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:09:07.947 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:09:07.985 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:09:08.026 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:09:08.187 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:09:08.238 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:09:08.292 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:09:08.366 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:09:08.397 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:09:08.434 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:09:08.480 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:09:08.508 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:09:08.550 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:09:08.593 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:09:08.729 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:09:08.782 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:09:08.812 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:09:08.853 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:09:08.948 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:09:08.979 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:09:09.010 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:09:09.036 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:09:09.061 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:09:09.136 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:09:09.262 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:09:09.310 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:09:09.337 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:09:09.372 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:09:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:09:09.443 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:09:09.473 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:09:09.499 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:09:09.532 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:09:09.574 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:09:09.608 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:09:09.695 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:09:09.738 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:09:09.770 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:09:09.817 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:09:09.916 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:09:09.972 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:09:10.696 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:09:10.735 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:09:10.769 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:09:10.866 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:09:10.927 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:09:11.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:09:11.055 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:09:11.231 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:09:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:09:11.338 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:09:11.981 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:09:12.091 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:09:13.319 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:09:13.398 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:09:13.426 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:09:13.454 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:09:13.753 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:09:13.789 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:09:13.878 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:09:13.910 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:09:13.946 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:09:14.425 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:09:14.459 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:09:14.521 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:09:14.544 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:09:16.804 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:09:16.840 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:09:17.380 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:09:17.427 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:09:17.508 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:09:17.508 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:09:17.508 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:09:17.508 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:17.537 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:09:29.218 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:09:29.218 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-04-23T18:09:29.219 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:29.400 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:09:29.415 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:09:29.415 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:29.415 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:09:29.415 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:30.903 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:09:30.928 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:09:30.928 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:30.928 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:09:30.928 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:09:30.928 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:09:30.929 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:31.161 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:09:31.183 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:09:31.183 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:31.183 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:09:31.184 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:09:31.184 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:09:31.184 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:31.629 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:09:31.651 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:09:31.651 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:31.651 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:09:31.651 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:09:31.651 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:09:31.652 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:31.759 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:09:31.815 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:09:31.871 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:09:32.059 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:09:32.105 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:09:32.183 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:09:32.272 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:09:32.412 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:09:32.455 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:09:32.486 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:09:32.514 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:09:32.908 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:09:32.938 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:09:32.970 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:09:33.032 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:09:33.061 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:09:34.097 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:09:34.130 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:09:35.098 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:09:35.122 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:09:35.223 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:09:35.387 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:09:35.416 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:09:35.438 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:09:35.438 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:35.439 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:09:35.439 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:09:35.439 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:09:35.439 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:35.483 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:09:35.494 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:09:35.591 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:09:35.648 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:09:36.570 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:09:36.603 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:09:36.624 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:09:36.625 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:36.625 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:09:36.625 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:09:36.625 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:09:36.625 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:36.669 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:09:36.691 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:09:36.691 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:36.691 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:09:36.691 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:36.737 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:09:36.759 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:09:36.759 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:36.759 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:09:36.759 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:09:36.759 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:09:36.759 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:37.070 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:09:37.093 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:09:37.093 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:09:37.093 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:09:37.093 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:09:37.093 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:09:37.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:41.699 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:09:41.741 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:09:41.782 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:09:41.813 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:09:41.840 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:09:42.027 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:09:42.070 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:09:42.100 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:09:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:09:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:09:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:09:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:09:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:09:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:09:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:09:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:09:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:09:43.966 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:09:43.991 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:09:43.991 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:09:44.080 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:09:44.147 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:09:49.806 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:09:49.807 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:09:49.808 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:09:49.809 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:09:49.810 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:09:49.811 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:09:49.812 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:09:49.813 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:09:49.814 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:09:49.815 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:09:49.816 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:09:49.817 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:09:49.818 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:09:49.819 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:09:49.819 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:09:49.819 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:09:51.896 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.897 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.898 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:09:51.899 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:09:51.900 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:09:51.901 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:09:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:09:51.903 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:09:51.904 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:09:51.905 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.906 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:09:51.907 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:09:51.908 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:09:51.909 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:09:51.909 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-23T18:09:51.909 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-04-23T18:09:52.293 DEBUG:teuthology.parallel:result is None 2024-04-23T18:09:52.294 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:09:52.446 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:09:52.473 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:09:52.473 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:09:52.473 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:09:52.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:09:52.623 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:09:52.644 INFO:teuthology.orchestra.run.smithi172.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:09:52.644 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:09:52.644 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:09:52.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:09:52.864 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:09:52.889 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:09:52.889 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:09:52.889 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:09:52.891 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T18:09:52.891 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:09:52.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:09:52.923 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:09:52.923 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:09:52.950 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:09:52.950 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:09:52.980 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T18:09:52.980 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:09:52.981 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:09:53.010 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:09:53.078 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:09:53.078 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:09:53.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:09:53.167 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:09:53.167 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:09:53.195 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:09:53.261 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T18:09:53.262 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:09:53.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:09:53.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:09:53.359 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:09:53.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:09:53.383 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:09:53.446 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:09:53.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:09:53.471 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:09:53.535 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T18:09:53.536 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:09:53.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:09:53.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:09:53.626 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:09:53.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:09:53.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:09:53.712 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:09:53.712 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:09:53.736 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:09:53.800 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T18:09:53.906 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-23T18:09:53.906 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:09:53.907 INFO:tasks.cephadm:Cluster fsid is af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:09:53.907 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T18:09:53.907 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.172', 'mon.c': '172.21.15.173'} 2024-04-23T18:09:53.907 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-04-23T18:09:53.907 INFO:tasks.cephadm:First mgr is x 2024-04-23T18:09:53.908 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T18:09:53.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-04-23T18:09:53.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-04-23T18:09:53.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-04-23T18:09:53.980 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T18:09:53.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:09:54.195 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-23T18:09:54.418 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:09:54.419 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-23T18:09:54.420 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:55.202 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:09 /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:55.203 DEBUG:teuthology.orchestra.run.smithi172:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:55.855 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:09 /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:55.855 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:56.361 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:09 /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:56.362 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:56.381 DEBUG:teuthology.orchestra.run.smithi172:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:56.399 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:09:56.436 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-23T18:09:56.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:09:56.439 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:09:56.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:09:57.904 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:09:58.016 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:09:58.052 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:10:26.918 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:10:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:10:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:10:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-04-23T18:10:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:10:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:10:26.919 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:10:29.816 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-23T18:10:29.816 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:10:29.816 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:10:29.816 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-04-23T18:10:29.817 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:10:29.817 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-23T18:10:29.817 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-23T18:10:31.809 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T18:10:31.810 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:10:31.810 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:10:31.810 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-04-23T18:10:31.810 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:10:31.810 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-04-23T18:10:31.810 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T18:10:31.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-04-23T18:10:31.867 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-04-23T18:10:31.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-04-23T18:10:31.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-04-23T18:10:31.955 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-04-23T18:10:31.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-04-23T18:10:32.004 INFO:tasks.cephadm:Writing seed config... 2024-04-23T18:10:32.005 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T18:10:32.006 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-23T18:10:32.007 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-23T18:10:32.008 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-23T18:10:32.009 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-23T18:10:32.010 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:10:32.010 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T18:10:32.028 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = af48a038-019c-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T18:10:32.029 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a.service 2024-04-23T18:10:32.072 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service 2024-04-23T18:10:32.115 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T18:10:32.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid af48a038-019c-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:10:32.319 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:10:32.319 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'af48a038-019c-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-04-23T18:10:32.320 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T18:10:32.320 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-04-23T18:10:32.346 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:10:32.346 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-04-23T18:10:32.346 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-04-23T18:10:32.353 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:10:32.353 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:10:32.359 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:10:32.359 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-23T18:10:32.366 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-23T18:10:32.372 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-23T18:10:32.372 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:10:32.372 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-04-23T18:10:32.397 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:10:32.398 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T18:10:32.398 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-04-23T18:10:32.398 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-04-23T18:10:32.403 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:10:32.404 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:10:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:10:32.411 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-23T18:10:32.416 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-23T18:10:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-23T18:10:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:10:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-04-23T18:10:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140278969203872 on /run/cephadm/af48a038-019c-11ef-bc93-c7b262605968.lock 2024-04-23T18:10:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140278969203872 acquired on /run/cephadm/af48a038-019c-11ef-bc93-c7b262605968.lock 2024-04-23T18:10:32.424 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-04-23T18:10:32.424 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-04-23T18:10:32.424 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-23T18:10:32.427 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-04-23T18:10:32.427 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-04-23T18:10:32.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T18:10:32.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-23T18:10:32.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-23T18:10:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T18:10:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T18:10:32.432 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:10:32.433 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-23T18:10:32.433 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-23T18:10:32.433 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:10:32.433 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-23T18:10:32.433 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-23T18:10:32.433 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T18:10:32.434 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T18:10:32.435 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:10:32.919 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T18:10:33.864 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:10:33.864 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:10:33.864 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T18:10:34.866 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-04-23T18:10:34.867 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-04-23T18:10:35.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCb+SdmuTdTCBAAldGmyTwoEI3B9jVdMU5dXg== 2024-04-23T18:10:36.636 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCc+SdmDRC4BxAAYY1oKkaSKMkkBu2FLZwNyg== 2024-04-23T18:10:37.518 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCc+Sdmb2pkORAA0yKnlofcAS3k9lFKK11SYg== 2024-04-23T18:10:37.519 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-04-23T18:10:38.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:10:38.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-23T18:10:38.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:10:38.367 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-04-23T18:10:39.338 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-04-23T18:10:39.566 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-23T18:10:39.718 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T18:10:39.896 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-af48a038-019c-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-af48a038-019c-11ef-bc93-c7b262605968.target. 2024-04-23T18:10:39.896 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-af48a038-019c-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-af48a038-019c-11ef-bc93-c7b262605968.target. 2024-04-23T18:10:40.157 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:10:40.157 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-23T18:10:40.326 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af48a038-019c-11ef-bc93-c7b262605968.target.wants/ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-af48a038-019c-11ef-bc93-c7b262605968@.service. 2024-04-23T18:10:40.875 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:10:40.875 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-23T18:10:40.883 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:10:40.883 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-23T18:10:40.883 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-23T18:10:40.884 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:10:40.884 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-04-23T18:10:40.884 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-04-23T18:10:40.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:40 smithi037 systemd[1]: Started Ceph mon.a for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:10:41.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T18:10:41.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:41.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T18:10:41.928 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.557445s) 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.313+0000 7fc062829700 1 Processor -- start 2024-04-23T18:10:41.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.313+0000 7fc062829700 1 -- start start 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.313+0000 7fc062829700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.313+0000 7fc062829700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c108f60 con 0x7fc05c108570 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.314+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.314+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41562/0 (socket says 172.21.15.37:41562) 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.314+0000 7fc05bfff700 1 -- 172.21.15.37:0/599041227 learned_addr learned my addr 172.21.15.37:0/599041227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.314+0000 7fc05bfff700 1 -- 172.21.15.37:0/599041227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c109720 con 0x7fc05c108570 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.314+0000 7fc05bfff700 1 --2- 172.21.15.37:0/599041227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c108990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc0500098a0 tx=0x7fc050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf54f1e5e65a03a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.314+0000 7fc05affd700 1 -- 172.21.15.37:0/599041227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc050004160 con 0x7fc05c108570 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc05affd700 1 -- 172.21.15.37:0/599041227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc050004300 con 0x7fc05c108570 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc05affd700 1 -- 172.21.15.37:0/599041227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0500036a0 con 0x7fc05c108570 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 -- 172.21.15.37:0/599041227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 msgr2=0x7fc05c108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:41.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 --2- 172.21.15.37:0/599041227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c108990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc0500098a0 tx=0x7fc050009bb0 comp rx=0 tx=0).stop 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 -- 172.21.15.37:0/599041227 shutdown_connections 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 --2- 172.21.15.37:0/599041227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c108990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 -- 172.21.15.37:0/599041227 >> 172.21.15.37:0/599041227 conn(0x7fc05c103b70 msgr2=0x7fc05c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 -- 172.21.15.37:0/599041227 shutdown_connections 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.315+0000 7fc062829700 1 -- 172.21.15.37:0/599041227 wait complete. 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc062829700 1 Processor -- start 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc062829700 1 -- start start 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc062829700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc062829700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c109c50 con 0x7fc05c108570 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41564/0 (socket says 172.21.15.37:41564) 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.316+0000 7fc05bfff700 1 -- 172.21.15.37:0/1197042668 learned_addr learned my addr 172.21.15.37:0/1197042668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc05bfff700 1 -- 172.21.15.37:0/1197042668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c19c1c0 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc05bfff700 1 --2- 172.21.15.37:0/1197042668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c19bc80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc050000c00 tx=0x7fc050004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc050003db0 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc050011630 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c19c450 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc050011950 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c19f140 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc05001a460 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.317+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc05001b780 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.318+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c0640a0 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.321+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc050031020 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.393+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc05c02e1f0 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.394+0000 7fc0597fa700 1 -- 172.21.15.37:0/1197042668 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc05001b980 con 0x7fc05c108570 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 msgr2=0x7fc05c19bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 --2- 172.21.15.37:0/1197042668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c19bc80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc050000c00 tx=0x7fc050004750 comp rx=0 tx=0).stop 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 shutdown_connections 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 --2- 172.21.15.37:0/1197042668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc05c108570 0x7fc05c19bc80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 >> 172.21.15.37:0/1197042668 conn(0x7fc05c103b70 msgr2=0x7fc05c192880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 shutdown_connections 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:41.395+0000 7fc062829700 1 -- 172.21.15.37:0/1197042668 wait complete. 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-04-23T18:10:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: monmap epoch 1 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: last_changed 2024-04-23T18:10:37.866561+0000 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: min_mon_release 19 (squid) 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: election_strategy: 1 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: fsmap 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: mgrmap e1: no daemons active 2024-04-23T18:10:42.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:41 smithi037 ceph-mon[29026]: from='client.? 172.21.15.37:0/1197042668' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T18:10:42.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:42.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:10:42.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:42.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:10:42.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f919290e700 1 Processor -- start 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f919290e700 1 -- start start 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f919290e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f919290e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f918c108f50 con 0x7f918c108560 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f918bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f918bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41570/0 (socket says 172.21.15.37:41570) 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f918bfff700 1 -- 172.21.15.37:0/1833462696 learned_addr learned my addr 172.21.15.37:0/1833462696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.350+0000 7f918bfff700 1 -- 172.21.15.37:0/1833462696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c109710 con 0x7f918c108560 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f918bfff700 1 --2- 172.21.15.37:0/1833462696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c108980 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098a0 tx=0x7f917c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e31e84ce8355d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f918affd700 1 -- 172.21.15.37:0/1833462696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c004160 con 0x7f918c108560 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f918affd700 1 -- 172.21.15.37:0/1833462696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c004300 con 0x7f918c108560 2024-04-23T18:10:42.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f918affd700 1 -- 172.21.15.37:0/1833462696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c0036a0 con 0x7f918c108560 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 -- 172.21.15.37:0/1833462696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 msgr2=0x7f918c108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 --2- 172.21.15.37:0/1833462696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c108980 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098a0 tx=0x7f917c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 -- 172.21.15.37:0/1833462696 shutdown_connections 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 --2- 172.21.15.37:0/1833462696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c108980 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 -- 172.21.15.37:0/1833462696 >> 172.21.15.37:0/1833462696 conn(0x7f918c103b40 msgr2=0x7f918c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 -- 172.21.15.37:0/1833462696 shutdown_connections 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.351+0000 7f919290e700 1 -- 172.21.15.37:0/1833462696 wait complete. 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f919290e700 1 Processor -- start 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f919290e700 1 -- start start 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f919290e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f919290e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f918c109c40 con 0x7f918c108560 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f918bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f918bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41582/0 (socket says 172.21.15.37:41582) 2024-04-23T18:10:42.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.352+0000 7f918bfff700 1 -- 172.21.15.37:0/1310012969 learned_addr learned my addr 172.21.15.37:0/1310012969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f918bfff700 1 -- 172.21.15.37:0/1310012969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c19c1e0 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f918bfff700 1 --2- 172.21.15.37:0/1310012969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c19bca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f917c000c00 tx=0x7f917c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c003db0 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c011630 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c19c470 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917c011950 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c19f160 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f917c01a460 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.353+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f917c01b780 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.354+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c0640a0 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.356+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f917c031020 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.427+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f918c02e1b0 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.430+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f917c01b980 con 0x7f918c108560 2024-04-23T18:10:42.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.431+0000 7f91897fa700 1 -- 172.21.15.37:0/1310012969 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f917c01a890 con 0x7f918c108560 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 msgr2=0x7f918c19bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 --2- 172.21.15.37:0/1310012969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c19bca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f917c000c00 tx=0x7f917c004750 comp rx=0 tx=0).stop 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 shutdown_connections 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 --2- 172.21.15.37:0/1310012969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f918c108560 0x7f918c19bca0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 >> 172.21.15.37:0/1310012969 conn(0x7f918c103b40 msgr2=0x7f918c1928c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 shutdown_connections 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:42.432+0000 7f919290e700 1 -- 172.21.15.37:0/1310012969 wait complete. 2024-04-23T18:10:42.974 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-04-23T18:10:43.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:42 smithi037 ceph-mon[29026]: from='client.? 172.21.15.37:0/1310012969' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:10:43.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:42 smithi037 ceph-mon[29026]: from='client.? 172.21.15.37:0/1310012969' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T18:10:44.000 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:43 smithi037 ceph-mon[29026]: from='client.? 172.21.15.37:0/1971696541' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:10:44.010 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b40f6700 1 Processor -- start 2024-04-23T18:10:44.010 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b40f6700 1 -- start start 2024-04-23T18:10:44.010 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b40f6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac10b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b40f6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10b8e0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b1e92700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac10b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b1e92700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac10b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41588/0 (socket says 172.21.15.37:41588) 2024-04-23T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.380+0000 7ff0b1e92700 1 -- 172.21.15.37:0/1623036533 learned_addr learned my addr 172.21.15.37:0/1623036533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.381+0000 7ff0b1e92700 1 -- 172.21.15.37:0/1623036533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac10ba60 con 0x7ff0ac108fb0 2024-04-23T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.381+0000 7ff0b1e92700 1 --2- 172.21.15.37:0/1623036533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac10b3a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80098a0 tx=0x7ff0a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3313f2d87909b3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:44.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.381+0000 7ff0b0e90700 1 -- 172.21.15.37:0/1623036533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a8004160 con 0x7ff0ac108fb0 2024-04-23T18:10:44.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.381+0000 7ff0b0e90700 1 -- 172.21.15.37:0/1623036533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff0a8004300 con 0x7ff0ac108fb0 2024-04-23T18:10:44.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.381+0000 7ff0b0e90700 1 -- 172.21.15.37:0/1623036533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a8011af0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1623036533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 msgr2=0x7ff0ac10b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:44.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 --2- 172.21.15.37:0/1623036533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac10b3a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80098a0 tx=0x7ff0a8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:10:44.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1623036533 shutdown_connections 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 --2- 172.21.15.37:0/1623036533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac10b3a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1623036533 >> 172.21.15.37:0/1623036533 conn(0x7ff0ac102a60 msgr2=0x7ff0ac104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1623036533 shutdown_connections 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1623036533 wait complete. 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.382+0000 7ff0b40f6700 1 Processor -- start 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b40f6700 1 -- start start 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b40f6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:44.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b40f6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10c9c0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b1e92700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:44.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b1e92700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41598/0 (socket says 172.21.15.37:41598) 2024-04-23T18:10:44.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b1e92700 1 -- 172.21.15.37:0/1971696541 learned_addr learned my addr 172.21.15.37:0/1971696541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:44.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b1e92700 1 -- 172.21.15.37:0/1971696541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac19cb30 con 0x7ff0ac108fb0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b1e92700 1 --2- 172.21.15.37:0/1971696541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac19c5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8011ac0 tx=0x7ff0a8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a8003970 con 0x7ff0ac108fb0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff0a8003b10 con 0x7ff0ac108fb0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac19cdc0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.383+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0a8024610 con 0x7ff0ac108fb0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.384+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac19fab0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.384+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0a8024bd0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.384+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0a8019700 con 0x7ff0ac108fb0 2024-04-23T18:10:44.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.384+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac0640a0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.387+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff0a8043e50 con 0x7ff0ac108fb0 2024-04-23T18:10:44.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.458+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff0ac19fff0 con 0x7ff0ac108fb0 2024-04-23T18:10:44.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.458+0000 7ff0a2ffd700 1 -- 172.21.15.37:0/1971696541 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff0a8032030 con 0x7ff0ac108fb0 2024-04-23T18:10:44.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 msgr2=0x7ff0ac19c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 --2- 172.21.15.37:0/1971696541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac19c5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8011ac0 tx=0x7ff0a8011ba0 comp rx=0 tx=0).stop 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 shutdown_connections 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 --2- 172.21.15.37:0/1971696541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0ac108fb0 0x7ff0ac19c5f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 >> 172.21.15.37:0/1971696541 conn(0x7ff0ac102a60 msgr2=0x7ff0ac103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 shutdown_connections 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:43.459+0000 7ff0b40f6700 1 -- 172.21.15.37:0/1971696541 wait complete. 2024-04-23T18:10:44.017 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-04-23T18:10:44.279 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:44 smithi037 systemd[1]: Stopping Ceph mon.a for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:10:44.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:44 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a[29021]: 2024-04-23T18:10:44.283+0000 7f10806fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:10:44.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:44 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a[29021]: 2024-04-23T18:10:44.283+0000 7f10806fe700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T18:10:44.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:44 smithi037 podman[29274]: 2024-04-23 18:10:44.448604943 +0000 UTC m=+0.273320205 container died 8dae73f543ade7d8ab205bdbc61adf2cad3b85c351ed21561b9392e86df8719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-23T18:10:45.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:44 smithi037 podman[29274]: 2024-04-23 18:10:44.850752996 +0000 UTC m=+0.675468251 container cleanup 8dae73f543ade7d8ab205bdbc61adf2cad3b85c351ed21561b9392e86df8719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:10:45.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:44 smithi037 bash[29274]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:10:45.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 podman[29288]: 2024-04-23 18:10:45.051422929 +0000 UTC m=+0.604043221 container remove 8dae73f543ade7d8ab205bdbc61adf2cad3b85c351ed21561b9392e86df8719b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T18:10:45.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-23T18:10:45.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 systemd[1]: Stopped Ceph mon.a for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:10:45.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 systemd[1]: Starting Ceph mon.a for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:10:45.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 podman[29385]: 2024-04-23 18:10:45.429878915 +0000 UTC m=+0.018469515 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:10:45.610 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 podman[29385]: 2024-04-23 18:10:45.527215105 +0000 UTC m=+0.115805687 container create 21630b3f612def12b1562f49f04d91e1cce3b7e55f632732aa1469a14313c397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:10:45.786 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 podman[29385]: 2024-04-23 18:10:45.712954751 +0000 UTC m=+0.301545332 container init 21630b3f612def12b1562f49f04d91e1cce3b7e55f632732aa1469a14313c397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 podman[29385]: 2024-04-23 18:10:45.716152479 +0000 UTC m=+0.304743060 container start 21630b3f612def12b1562f49f04d91e1cce3b7e55f632732aa1469a14313c397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: pidfile_write: ignore empty --pid-file 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: load: jerasure load: lrc 2024-04-23T18:10:46.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Git sha 0 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: DB SUMMARY 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: DB Session ID: LBMY3ACAKSMTYVZPHNC6 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105442 ; 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:10:46.011 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.env: 0x56130cf5dcc0 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.info_log: 0x56130e179be0 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.statistics: (nil) 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.use_fsync: 0 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:10:46.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.db_log_dir: 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.wal_dir: 2024-04-23T18:10:46.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.write_buffer_manager: 0x56130e10d0e0 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:10:46.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.unordered_write: 0 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.row_cache: None 2024-04-23T18:10:46.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.wal_filter: None 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.wal_compression: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:10:46.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:10:46.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_open_files: -1 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:10:46.019 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Compression algorithms supported: 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kZSTD supported: 0 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:10:46.020 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.merge_operator: 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_filter: None 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56130e179dc0) 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:10:46.021 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x56130e13f1f0 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-04-23T18:10:46.022 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-04-23T18:10:46.023 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:10:46.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression: NoCompression 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.num_levels: 7 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:10:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:10:46.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:10:46.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:10:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:10:46.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:10:46.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.ttl: 2592000 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:10:46.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 787a47e8-8dd2-47a8-84da-151d64a988bf 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713895845745490, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713895845746583, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98439, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90693, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713895845, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "787a47e8-8dd2-47a8-84da-151d64a988bf", "db_session_id": "LBMY3ACAKSMTYVZPHNC6", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713895845746668, "job": 1, "event": "recovery_finished"} 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T18:10:46.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56130e254000 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: DB pointer 0x56130e23e000 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:10:46.033 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 100.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 100.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:10:46.034 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 30.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 30.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x56130e13f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-23T18:10:46.035 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???) e1 preinit fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).mds e1 new map 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).mds e1 print_map 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-04-23T18:10:46.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 bash[29385]: 21630b3f612def12b1562f49f04d91e1cce3b7e55f632732aa1469a14313c397 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 systemd[1]: Started Ceph mon.a for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: monmap epoch 1 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: last_changed 2024-04-23T18:10:37.866561+0000 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: min_mon_release 19 (squid) 2024-04-23T18:10:46.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: election_strategy: 1 2024-04-23T18:10:46.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:10:46.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: fsmap 2024-04-23T18:10:46.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:10:46.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:45 smithi037 ceph-mon[29404]: mgrmap e1: no daemons active 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.227+0000 7f2346589700 1 Processor -- start 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.227+0000 7f2346589700 1 -- start start 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.227+0000 7f2346589700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f2340108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.227+0000 7f2346589700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2340108f50 con 0x7f2340108560 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f2340108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f2340108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33654/0 (socket says 172.21.15.37:33654) 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233ffff700 1 -- 172.21.15.37:0/356993857 learned_addr learned my addr 172.21.15.37:0/356993857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233ffff700 1 -- 172.21.15.37:0/356993857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2340109710 con 0x7f2340108560 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233ffff700 1 --2- 172.21.15.37:0/356993857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f2340108980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f23300098a0 tx=0x7f2330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11be82579df0e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:46.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233effd700 1 -- 172.21.15.37:0/356993857 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2330004160 con 0x7f2340108560 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.228+0000 7f233effd700 1 -- 172.21.15.37:0/356993857 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2330004300 con 0x7f2340108560 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f233effd700 1 -- 172.21.15.37:0/356993857 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2330011af0 con 0x7f2340108560 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 -- 172.21.15.37:0/356993857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 msgr2=0x7f2340108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 --2- 172.21.15.37:0/356993857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f2340108980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f23300098a0 tx=0x7f2330009bb0 comp rx=0 tx=0).stop 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 -- 172.21.15.37:0/356993857 shutdown_connections 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 --2- 172.21.15.37:0/356993857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f2340108980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 -- 172.21.15.37:0/356993857 >> 172.21.15.37:0/356993857 conn(0x7f2340103b40 msgr2=0x7f2340105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 -- 172.21.15.37:0/356993857 shutdown_connections 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.229+0000 7f2346589700 1 -- 172.21.15.37:0/356993857 wait complete. 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f2346589700 1 Processor -- start 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f2346589700 1 -- start start 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f2346589700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f234019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:46.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f2346589700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f234010cac0 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f233ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f234019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f233ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f234019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33656/0 (socket says 172.21.15.37:33656) 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f233ffff700 1 -- 172.21.15.37:0/3927580033 learned_addr learned my addr 172.21.15.37:0/3927580033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.230+0000 7f233ffff700 1 -- 172.21.15.37:0/3927580033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234019ca50 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f233ffff700 1 --2- 172.21.15.37:0/3927580033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f234019c510 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f23300117b0 tx=0x7f2330011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2330003af0 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2330020070 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234019cce0 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2330019630 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234019f9d0 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2330019c30 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.231+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f233000b720 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.232+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23400640a0 con 0x7f2340108560 2024-04-23T18:10:46.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.234+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f23300042a0 con 0x7f2340108560 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.305+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f234019fba0 con 0x7f2340108560 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.307+0000 7f233d7fa700 1 -- 172.21.15.37:0/3927580033 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f233001a9c0 con 0x7f2340108560 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 msgr2=0x7f234019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 --2- 172.21.15.37:0/3927580033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f234019c510 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f23300117b0 tx=0x7f2330011b70 comp rx=0 tx=0).stop 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 shutdown_connections 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 --2- 172.21.15.37:0/3927580033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340108560 0x7f234019c510 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 >> 172.21.15.37:0/3927580033 conn(0x7f2340103b40 msgr2=0x7f23401046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 shutdown_connections 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:46.308+0000 7f2346589700 1 -- 172.21.15.37:0/3927580033 wait complete. 2024-04-23T18:10:46.831 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T18:10:46.845 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:10:46.845 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-04-23T18:10:46.845 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T18:10:46.846 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T18:10:47.090 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:10:47.091 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-23T18:10:47.246 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af48a038-019c-11ef-bc93-c7b262605968.target.wants/ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-af48a038-019c-11ef-bc93-c7b262605968@.service. 2024-04-23T18:10:47.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:47 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3927580033' entity='client.admin' 2024-04-23T18:10:47.733 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:10:47.734 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-23T18:10:47.740 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:10:47.740 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-23T18:10:47.740 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-23T18:10:47.740 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:10:47.750 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:10:47.750 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-23T18:10:47.759 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:10:47.759 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-23T18:10:47.760 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-23T18:10:47.760 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-23T18:10:47.760 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-04-23T18:10:47.760 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-04-23T18:10:48.205 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:48.126+0000 7f7d49043200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:10:48.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:48 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2842696873' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:10:48.539 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:48.456+0000 7f7d49043200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:10:48.539 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:48.543+0000 7f7d49043200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:10:50.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:49.804+0000 7f7d49043200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:10:50.122 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:49.980+0000 7f7d49043200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:10:50.452 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:50.157+0000 7f7d49043200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:10:50.452 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:50.458+0000 7f7d49043200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:10:50.758 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:50.553+0000 7f7d49043200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:10:50.758 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:50.764+0000 7f7d49043200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:10:51.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:50.982+0000 7f7d49043200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:10:51.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:10:51.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:10:51.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:10:51.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:10:51.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:10:51.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:10:51.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:10:40.840434+0000", 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.220+0000 7f7b764ab700 1 Processor -- start 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.220+0000 7f7b764ab700 1 -- start start 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b764ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b70107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b764ab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70107e30 con 0x7f7b70107440 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b70107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b70107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33674/0 (socket says 172.21.15.37:33674) 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b6ffff700 1 -- 172.21.15.37:0/1497953704 learned_addr learned my addr 172.21.15.37:0/1497953704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b6ffff700 1 -- 172.21.15.37:0/1497953704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b70108640 con 0x7f7b70107440 2024-04-23T18:10:51.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.221+0000 7f7b6ffff700 1 --2- 172.21.15.37:0/1497953704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b70107860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7b600098a0 tx=0x7f7b60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e23af2559137e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b6effd700 1 -- 172.21.15.37:0/1497953704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b60004160 con 0x7f7b70107440 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b6effd700 1 -- 172.21.15.37:0/1497953704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b60004300 con 0x7f7b70107440 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b6effd700 1 -- 172.21.15.37:0/1497953704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b60011af0 con 0x7f7b70107440 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 -- 172.21.15.37:0/1497953704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 msgr2=0x7f7b70107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 --2- 172.21.15.37:0/1497953704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b70107860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7b600098a0 tx=0x7f7b60009bb0 comp rx=0 tx=0).stop 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 -- 172.21.15.37:0/1497953704 shutdown_connections 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 --2- 172.21.15.37:0/1497953704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b70107860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 -- 172.21.15.37:0/1497953704 >> 172.21.15.37:0/1497953704 conn(0x7f7b70102a40 msgr2=0x7f7b70104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 -- 172.21.15.37:0/1497953704 shutdown_connections 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.222+0000 7f7b764ab700 1 -- 172.21.15.37:0/1497953704 wait complete. 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.223+0000 7f7b764ab700 1 Processor -- start 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.223+0000 7f7b764ab700 1 -- start start 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b764ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b7019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:51.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b764ab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7010b9f0 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b7019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b7019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33678/0 (socket says 172.21.15.37:33678) 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b6ffff700 1 -- 172.21.15.37:0/2842696873 learned_addr learned my addr 172.21.15.37:0/2842696873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b6ffff700 1 -- 172.21.15.37:0/2842696873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7019b910 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.224+0000 7f7b6ffff700 1 --2- 172.21.15.37:0/2842696873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b7019b3d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7b60003f40 tx=0x7f7b60011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b600038b0 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b60003a50 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7019bba0 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b6001a5c0 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7019e890 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b6001ac50 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.225+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b60019700 con 0x7f7b70107440 2024-04-23T18:10:51.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.226+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b700640a0 con 0x7f7b70107440 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.230+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7b60043e50 con 0x7f7b70107440 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.303+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7b7019ea60 con 0x7f7b70107440 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.304+0000 7f7b6d7fa700 1 -- 172.21.15.37:0/2842696873 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7b7019ea60 con 0x7f7b70107440 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 msgr2=0x7f7b7019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 --2- 172.21.15.37:0/2842696873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b7019b3d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7b60003f40 tx=0x7f7b60011890 comp rx=0 tx=0).stop 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 shutdown_connections 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 --2- 172.21.15.37:0/2842696873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70107440 0x7f7b7019b3d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 >> 172.21.15.37:0/2842696873 conn(0x7f7b70102a40 msgr2=0x7f7b701035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 shutdown_connections 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:48.305+0000 7f7b764ab700 1 -- 172.21.15.37:0/2842696873 wait complete. 2024-04-23T18:10:51.767 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-04-23T18:10:51.871 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:51.610+0000 7f7d49043200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:10:52.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:52.050+0000 7f7d49043200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:10:52.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:52.151+0000 7f7d49043200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:10:52.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:52.360+0000 7f7d49043200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:10:53.244 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:52.901+0000 7f7d49043200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:10:53.244 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:52.997+0000 7f7d49043200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:10:53.603 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:53.250+0000 7f7d49043200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:10:53.603 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:53.388+0000 7f7d49043200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:10:53.603 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:53.487+0000 7f7d49043200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:10:53.603 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:53.608+0000 7f7d49043200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:10:54.161 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:53.925+0000 7f7d49043200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:10:54.727 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:54 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1969175787' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:10:54.727 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:54.568+0000 7f7d49043200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:10:54.986 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:10:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:10:54.793+0000 7f7d49043200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:10:54.989 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:10:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:10:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:10:55.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:10:55.001 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:10:55.002 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:55.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:10:40.840434+0000", 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f2075322700 1 Processor -- start 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f2075322700 1 -- start start 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f2075322700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f207010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f2075322700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f207007bec0 con 0x7f2070108fb0 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f206effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f207010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f206effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f207010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33692/0 (socket says 172.21.15.37:33692) 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.357+0000 7f206effd700 1 -- 172.21.15.37:0/3936769419 learned_addr learned my addr 172.21.15.37:0/3936769419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:55.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f206effd700 1 -- 172.21.15.37:0/3936769419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207010b8e0 con 0x7f2070108fb0 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f206effd700 1 --2- 172.21.15.37:0/3936769419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f207010b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20640098a0 tx=0x7f2064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9642410f56a7efb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f206dffb700 1 -- 172.21.15.37:0/3936769419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2064004160 con 0x7f2070108fb0 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f206dffb700 1 -- 172.21.15.37:0/3936769419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2064004300 con 0x7f2070108fb0 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f206dffb700 1 -- 172.21.15.37:0/3936769419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2064011af0 con 0x7f2070108fb0 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f2075322700 1 -- 172.21.15.37:0/3936769419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 msgr2=0x7f207010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.358+0000 7f2075322700 1 --2- 172.21.15.37:0/3936769419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f207010b3a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f20640098a0 tx=0x7f2064009bb0 comp rx=0 tx=0).stop 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 -- 172.21.15.37:0/3936769419 shutdown_connections 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 --2- 172.21.15.37:0/3936769419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f207010b3a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 -- 172.21.15.37:0/3936769419 >> 172.21.15.37:0/3936769419 conn(0x7f2070102a40 msgr2=0x7f2070104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 -- 172.21.15.37:0/3936769419 shutdown_connections 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 -- 172.21.15.37:0/3936769419 wait complete. 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 Processor -- start 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.359+0000 7f2075322700 1 -- start start 2024-04-23T18:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2075322700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f2070108830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2075322700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f207010c8b0 con 0x7f2070108fb0 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f206effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f2070108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f206effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f2070108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33708/0 (socket says 172.21.15.37:33708) 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f206effd700 1 -- 172.21.15.37:0/1969175787 learned_addr learned my addr 172.21.15.37:0/1969175787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f206effd700 1 -- 172.21.15.37:0/1969175787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2070108d70 con 0x7f2070108fb0 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f206effd700 1 --2- 172.21.15.37:0/1969175787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f2070108830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2064009fb0 tx=0x7f2064011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20640036a0 con 0x7f2070108fb0 2024-04-23T18:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2070106f60 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f206401a430 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2064019550 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.360+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20701074a0 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.361+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2064019be0 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.361+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2064024730 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.361+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20700640a0 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.363+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2064043e50 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.437+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2070107750 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.438+0000 7f2057fff700 1 -- 172.21.15.37:0/1969175787 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2064032070 con 0x7f2070108fb0 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 msgr2=0x7f2070108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 --2- 172.21.15.37:0/1969175787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f2070108830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2064009fb0 tx=0x7f2064011940 comp rx=0 tx=0).stop 2024-04-23T18:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 shutdown_connections 2024-04-23T18:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 --2- 172.21.15.37:0/1969175787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2070108fb0 0x7f2070108830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 >> 172.21.15.37:0/1969175787 conn(0x7f2070102a40 msgr2=0x7f20701035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 shutdown_connections 2024-04-23T18:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:54.439+0000 7f2075322700 1 -- 172.21.15.37:0/1969175787 wait complete. 2024-04-23T18:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: Activating manager daemon x 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: mgrmap e2: x(active, starting, since 0.00289874s) 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: Manager daemon x is now available 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' 2024-04-23T18:10:55.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' 2024-04-23T18:10:55.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:55 smithi037 ceph-mon[29404]: from='mgr.14100 172.21.15.37:0/602442254' entity='mgr.x' 2024-04-23T18:10:57.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:56 smithi037 ceph-mon[29404]: mgrmap e3: x(active, since 1.00767s) 2024-04-23T18:10:57.978 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:57 smithi037 ceph-mon[29404]: mgrmap e4: x(active, since 2s) 2024-04-23T18:10:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:10:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3716274823' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:11:04.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:11:04.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:11:04.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:11:04.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:11:04.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:11:04.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:11:04.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:11:04.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:11:04.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:11:04.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:11:04.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:11:04.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:11:04.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:11:04.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:11:04.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:10:40.840434+0000", 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.673+0000 7f15b9dc1700 1 Processor -- start 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b9dc1700 1 -- start start 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b9dc1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b410b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b9dc1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15b407be80 con 0x7f15b4109000 2024-04-23T18:11:04.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b410b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b410b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50698/0 (socket says 172.21.15.37:50698) 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b37fe700 1 -- 172.21.15.37:0/1430826366 learned_addr learned my addr 172.21.15.37:0/1430826366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b37fe700 1 -- 172.21.15.37:0/1430826366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b410b930 con 0x7f15b4109000 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.674+0000 7f15b37fe700 1 --2- 172.21.15.37:0/1430826366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b410b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098a0 tx=0x7f15a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2140b67bbddb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b27fc700 1 -- 172.21.15.37:0/1430826366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a4004160 con 0x7f15b4109000 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b27fc700 1 -- 172.21.15.37:0/1430826366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f15a4004300 con 0x7f15b4109000 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b27fc700 1 -- 172.21.15.37:0/1430826366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a4011af0 con 0x7f15b4109000 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b9dc1700 1 -- 172.21.15.37:0/1430826366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 msgr2=0x7f15b410b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b9dc1700 1 --2- 172.21.15.37:0/1430826366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b410b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098a0 tx=0x7f15a4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b9dc1700 1 -- 172.21.15.37:0/1430826366 shutdown_connections 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b9dc1700 1 --2- 172.21.15.37:0/1430826366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b410b3f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b9dc1700 1 -- 172.21.15.37:0/1430826366 >> 172.21.15.37:0/1430826366 conn(0x7f15b4102a50 msgr2=0x7f15b4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.675+0000 7f15b9dc1700 1 -- 172.21.15.37:0/1430826366 shutdown_connections 2024-04-23T18:11:04.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.676+0000 7f15b9dc1700 1 -- 172.21.15.37:0/1430826366 wait complete. 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.676+0000 7f15b9dc1700 1 Processor -- start 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.676+0000 7f15b9dc1700 1 -- start start 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.676+0000 7f15b9dc1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b4108880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.676+0000 7f15b9dc1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15b410c8f0 con 0x7f15b4109000 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b4108880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b4108880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50700/0 (socket says 172.21.15.37:50700) 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b37fe700 1 -- 172.21.15.37:0/3716274823 learned_addr learned my addr 172.21.15.37:0/3716274823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b37fe700 1 -- 172.21.15.37:0/3716274823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b4108dc0 con 0x7f15b4109000 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b37fe700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b4108880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009fb0 tx=0x7f15a4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a40036a0 con 0x7f15b4109000 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f15a401a430 con 0x7f15b4109000 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a4019550 con 0x7f15b4109000 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b4106fb0 con 0x7f15b4109000 2024-04-23T18:11:04.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.677+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41074f0 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.678+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f15a401a8e0 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.678+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b40640a0 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.679+0000 7f15b0ff9700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f159c039510 0x7f159c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.679+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f15a404d870 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.679+0000 7f15b2ffd700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f159c039510 0x7f159c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.679+0000 7f15b2ffd700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f159c039510 0x7f159c03b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.684+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15a40172b0 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.983+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f15b4107770 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.984+0000 7f15b0ff9700 1 -- 172.21.15.37:0/3716274823 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f15a401a690 con 0x7f15b4109000 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f159c039510 msgr2=0x7f159c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f159c039510 0x7f159c03b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 msgr2=0x7f15b4108880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:04.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b4108880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009fb0 tx=0x7f15a4011940 comp rx=0 tx=0).stop 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 shutdown_connections 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f159c039510 0x7f159c03b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 --2- 172.21.15.37:0/3716274823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b4109000 0x7f15b4108880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.987+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 >> 172.21.15.37:0/3716274823 conn(0x7f15b4102a50 msgr2=0x7f15b41035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.988+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 shutdown_connections 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:10:57.988+0000 7f15b9dc1700 1 -- 172.21.15.37:0/3716274823 wait complete. 2024-04-23T18:11:04.234 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-04-23T18:11:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:04 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3980822983' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:11:05.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:11:05.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:11:05.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:11:05.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:11:05.426 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ec6c700 1 Processor -- start 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ec6c700 1 -- start start 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ec6c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc798108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ec6c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc798108f50 con 0x7fc798108560 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ca08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc798108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ca08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc798108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40352/0 (socket says 172.21.15.37:40352) 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.661+0000 7fc79ca08700 1 -- 172.21.15.37:0/1285054701 learned_addr learned my addr 172.21.15.37:0/1285054701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.662+0000 7fc79ca08700 1 -- 172.21.15.37:0/1285054701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc798109710 con 0x7fc798108560 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.662+0000 7fc79ca08700 1 --2- 172.21.15.37:0/1285054701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc798108980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098a0 tx=0x7fc78c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab040def82505257 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.662+0000 7fc7977fe700 1 -- 172.21.15.37:0/1285054701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c004160 con 0x7fc798108560 2024-04-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.662+0000 7fc7977fe700 1 -- 172.21.15.37:0/1285054701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc78c004300 con 0x7fc798108560 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.662+0000 7fc7977fe700 1 -- 172.21.15.37:0/1285054701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c011af0 con 0x7fc798108560 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- 172.21.15.37:0/1285054701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 msgr2=0x7fc798108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 --2- 172.21.15.37:0/1285054701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc798108980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098a0 tx=0x7fc78c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- 172.21.15.37:0/1285054701 shutdown_connections 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 --2- 172.21.15.37:0/1285054701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc798108980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- 172.21.15.37:0/1285054701 >> 172.21.15.37:0/1285054701 conn(0x7fc798103b40 msgr2=0x7fc798105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- 172.21.15.37:0/1285054701 shutdown_connections 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- 172.21.15.37:0/1285054701 wait complete. 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 Processor -- start 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- start start 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc79819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.663+0000 7fc79ec6c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc79810cac0 con 0x7fc798108560 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ca08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc79819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:05.428 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ca08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc79819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40354/0 (socket says 172.21.15.37:40354) 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ca08700 1 -- 172.21.15.37:0/3980822983 learned_addr learned my addr 172.21.15.37:0/3980822983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ca08700 1 -- 172.21.15.37:0/3980822983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79819ca50 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ca08700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc79819c510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009fb0 tx=0x7fc78c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c0036a0 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc78c01a430 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79819cce0 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c019550 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.664+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79819f9d0 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.665+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc78c019800 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.665+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7980640a0 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.665+0000 7fc795ffb700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fc788039560 0x7fc78803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.665+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc78c04d9a0 con 0x7fc798108560 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.666+0000 7fc797fff700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fc788039560 0x7fc78803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:05.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.666+0000 7fc797fff700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fc788039560 0x7fc78803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc784006fd0 tx=0x7fc784006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.671+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc78c04d3c0 con 0x7fc798108560 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.876+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc79819fba0 con 0x7fc798108560 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.877+0000 7fc795ffb700 1 -- 172.21.15.37:0/3980822983 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fc79819fba0 con 0x7fc798108560 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fc788039560 msgr2=0x7fc78803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fc788039560 0x7fc78803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc784006fd0 tx=0x7fc784006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 msgr2=0x7fc79819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc79819c510 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009fb0 tx=0x7fc78c011940 comp rx=0 tx=0).stop 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 shutdown_connections 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fc788039560 0x7fc78803ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:05.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 --2- 172.21.15.37:0/3980822983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc798108560 0x7fc79819c510 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:05.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.879+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 >> 172.21.15.37:0/3980822983 conn(0x7fc798103b40 msgr2=0x7fc7981046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:05.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.880+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 shutdown_connections 2024-04-23T18:11:05.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:04.880+0000 7fc79ec6c700 1 -- 172.21.15.37:0/3980822983 wait complete. 2024-04-23T18:11:05.431 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-04-23T18:11:06.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:06 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/648271662' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T18:11:06.504 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setuser ceph since I am not root 2024-04-23T18:11:06.504 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setgroup ceph since I am not root 2024-04-23T18:11:06.504 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:06.383+0000 7fb7619ef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:11:06.787 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:06.575+0000 7fb7619ef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:11:06.787 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:06.681+0000 7fb7619ef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:11:06.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb576e8e700 1 Processor -- start 2024-04-23T18:11:06.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb576e8e700 1 -- start start 2024-04-23T18:11:06.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb576e8e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:06.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb576e8e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb57007c900 con 0x7fb57007de80 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb574c2a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb574c2a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40356/0 (socket says 172.21.15.37:40356) 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb574c2a700 1 -- 172.21.15.37:0/1467523738 learned_addr learned my addr 172.21.15.37:0/1467523738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.880+0000 7fb574c2a700 1 -- 172.21.15.37:0/1467523738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57007ca80 con 0x7fb57007de80 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb574c2a700 1 --2- 172.21.15.37:0/1467523738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57007c330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5640098a0 tx=0x7fb564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286f38e01a41bd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb56f7fe700 1 -- 172.21.15.37:0/1467523738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb564004160 con 0x7fb57007de80 2024-04-23T18:11:06.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb56f7fe700 1 -- 172.21.15.37:0/1467523738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb564004300 con 0x7fb57007de80 2024-04-23T18:11:06.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb56f7fe700 1 -- 172.21.15.37:0/1467523738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb564011af0 con 0x7fb57007de80 2024-04-23T18:11:06.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb576e8e700 1 -- 172.21.15.37:0/1467523738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 msgr2=0x7fb57007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:06.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb576e8e700 1 --2- 172.21.15.37:0/1467523738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57007c330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5640098a0 tx=0x7fb564009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:06.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.881+0000 7fb576e8e700 1 -- 172.21.15.37:0/1467523738 shutdown_connections 2024-04-23T18:11:06.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 --2- 172.21.15.37:0/1467523738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57007c330 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:06.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 -- 172.21.15.37:0/1467523738 >> 172.21.15.37:0/1467523738 conn(0x7fb570103b40 msgr2=0x7fb570105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:06.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 -- 172.21.15.37:0/1467523738 shutdown_connections 2024-04-23T18:11:06.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 -- 172.21.15.37:0/1467523738 wait complete. 2024-04-23T18:11:06.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 Processor -- start 2024-04-23T18:11:06.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 -- start start 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.882+0000 7fb576e8e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010ea90 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb574c2a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb574c2a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40362/0 (socket says 172.21.15.37:40362) 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb574c2a700 1 -- 172.21.15.37:0/648271662 learned_addr learned my addr 172.21.15.37:0/648271662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb574c2a700 1 -- 172.21.15.37:0/648271662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57019ec00 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb574c2a700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57019e6c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fb0 tx=0x7fb564011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5640036a0 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb56401a430 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb564019550 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.883+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57019ee90 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.884+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a1b80 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.884+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fb56401a8e0 con 0x7fb57007de80 2024-04-23T18:11:06.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.884+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5700640a0 con 0x7fb57007de80 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.885+0000 7fb56dffb700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fb560039560 0x7fb56003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.885+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb56404d840 con 0x7fb57007de80 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.885+0000 7fb56ffff700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fb560039560 0x7fb56003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.885+0000 7fb56ffff700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fb560039560 0x7fb56003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006fd0 tx=0x7fb55c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:05.890+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb56401a690 con 0x7fb57007de80 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.159+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb5701a1d50 con 0x7fb57007de80 2024-04-23T18:11:06.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.211+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb5701a1d50 con 0x7fb57007de80 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.212+0000 7fb56dffb700 1 -- 172.21.15.37:0/648271662 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb564003840 con 0x7fb57007de80 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fb560039560 msgr2=0x7fb56003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fb560039560 0x7fb56003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006fd0 tx=0x7fb55c006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 msgr2=0x7fb57019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57019e6c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fb0 tx=0x7fb564011940 comp rx=0 tx=0).stop 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 shutdown_connections 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fb560039560 0x7fb56003ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.213+0000 7fb576e8e700 1 --2- 172.21.15.37:0/648271662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb57007de80 0x7fb57019e6c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.214+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 >> 172.21.15.37:0/648271662 conn(0x7fb570103b40 msgr2=0x7fb570104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.214+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 shutdown_connections 2024-04-23T18:11:06.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:06.214+0000 7fb576e8e700 1 -- 172.21.15.37:0/648271662 wait complete. 2024-04-23T18:11:07.056 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:07.008+0000 7fb7619ef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:11:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:07 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/648271662' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T18:11:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:07 smithi037 ceph-mon[29404]: mgrmap e5: x(active, since 11s) 2024-04-23T18:11:07.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:07.098+0000 7fb7619ef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.256+0000 7fa5a384e700 1 Processor -- start 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.256+0000 7fa5a384e700 1 -- start start 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.256+0000 7fa5a384e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c106c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.256+0000 7fa5a384e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c107210 con 0x7fa59c106820 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.256+0000 7fa5a15ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:08.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa5a15ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c106c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40382/0 (socket says 172.21.15.37:40382) 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa5a15ea700 1 -- 172.21.15.37:0/831752684 learned_addr learned my addr 172.21.15.37:0/831752684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa5a15ea700 1 -- 172.21.15.37:0/831752684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c107590 con 0x7fa59c106820 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa5a15ea700 1 --2- 172.21.15.37:0/831752684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c106c40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098a0 tx=0x7fa58c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e53bbb5dd442e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa593fff700 1 -- 172.21.15.37:0/831752684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa58c004160 con 0x7fa59c106820 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa593fff700 1 -- 172.21.15.37:0/831752684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa58c004300 con 0x7fa59c106820 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.257+0000 7fa593fff700 1 -- 172.21.15.37:0/831752684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa58c011af0 con 0x7fa59c106820 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 -- 172.21.15.37:0/831752684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 msgr2=0x7fa59c106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 --2- 172.21.15.37:0/831752684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c106c40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098a0 tx=0x7fa58c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 -- 172.21.15.37:0/831752684 shutdown_connections 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 --2- 172.21.15.37:0/831752684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c106c40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 -- 172.21.15.37:0/831752684 >> 172.21.15.37:0/831752684 conn(0x7fa59c1021a0 msgr2=0x7fa59c1045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 -- 172.21.15.37:0/831752684 shutdown_connections 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.258+0000 7fa5a384e700 1 -- 172.21.15.37:0/831752684 wait complete. 2024-04-23T18:11:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a384e700 1 Processor -- start 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a384e700 1 -- start start 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a384e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a384e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c108950 con 0x7fa59c106820 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a15ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a15ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40398/0 (socket says 172.21.15.37:40398) 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a15ea700 1 -- 172.21.15.37:0/2725571254 learned_addr learned my addr 172.21.15.37:0/2725571254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.259+0000 7fa5a15ea700 1 -- 172.21.15.37:0/2725571254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c19ebf0 con 0x7fa59c106820 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.260+0000 7fa5a15ea700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c19e6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa58c003f40 tx=0x7fa58c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.260+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa58c0038b0 con 0x7fa59c106820 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.260+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa58c003a50 con 0x7fa59c106820 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.260+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c19ee80 con 0x7fa59c106820 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.260+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa58c01a5c0 con 0x7fa59c106820 2024-04-23T18:11:08.130 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.260+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1a1b20 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.261+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fa58c01a8c0 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.261+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c0640a0 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.261+0000 7fa5927fc700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 0x7fa58803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.261+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa58c04db10 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.261+0000 7fa5a0de9700 1 -- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 msgr2=0x7fa58803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.261+0000 7fa5a0de9700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 0x7fa58803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.266+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa58c019c70 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.462+0000 7fa5a0de9700 1 -- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 msgr2=0x7fa58803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.462+0000 7fa5a0de9700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 0x7fa58803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.584+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa59c1a1cf0 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.584+0000 7fa5927fc700 1 -- 172.21.15.37:0/2725571254 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa59c1a1cf0 con 0x7fa59c106820 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 msgr2=0x7fa58803ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 0x7fa58803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:08.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 msgr2=0x7fa59c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c19e6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa58c003f40 tx=0x7fa58c011890 comp rx=0 tx=0).stop 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 shutdown_connections 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7fa588039560 0x7fa58803ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 --2- 172.21.15.37:0/2725571254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa59c106820 0x7fa59c19e6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 >> 172.21.15.37:0/2725571254 conn(0x7fa59c1021a0 msgr2=0x7fa59c194cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 shutdown_connections 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:07.586+0000 7fa5a384e700 1 -- 172.21.15.37:0/2725571254 wait complete. 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-23T18:11:08.132 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-04-23T18:11:08.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:08 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2725571254' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:11:08.471 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:08.359+0000 7fb7619ef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:11:08.781 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:08.544+0000 7fb7619ef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:11:08.781 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:08.723+0000 7fb7619ef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:11:09.114 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:09.024+0000 7fb7619ef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:11:09.114 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:09.120+0000 7fb7619ef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:11:09.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:09.330+0000 7fb7619ef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:11:09.871 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:09.531+0000 7fb7619ef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:11:10.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:10.154+0000 7fb7619ef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:11:10.871 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:10.593+0000 7fb7619ef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:11:10.871 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:10.693+0000 7fb7619ef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:11:11.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:10.902+0000 7fb7619ef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:11:11.788 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:11.445+0000 7fb7619ef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:11:11.788 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:11.541+0000 7fb7619ef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:11:12.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:11.794+0000 7fb7619ef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:11:12.122 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:11.932+0000 7fb7619ef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:11:12.122 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:12.031+0000 7fb7619ef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:11:12.456 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:12.151+0000 7fb7619ef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:11:12.456 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:12.461+0000 7fb7619ef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:11:13.360 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:13.088+0000 7fb7619ef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:11:13.360 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:13.313+0000 7fb7619ef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:11:13.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: Active manager daemon x restarted 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: Activating manager daemon x 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: mgrmap e6: x(active, starting, since 0.00341891s) 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: Manager daemon x is now available 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:13.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:13.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:13 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:11:14.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:14 smithi037 ceph-mon[29404]: Found migration_current of "None". Setting to last migration. 2024-04-23T18:11:14.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:14 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:11:14.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:14 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:14.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:14 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:14.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:14 smithi037 ceph-mon[29404]: mgrmap e7: x(active, since 1.0078s) 2024-04-23T18:11:14.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:11:14.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.581+0000 7f987e836700 1 Processor -- start 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.581+0000 7f987e836700 1 -- start start 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f987e836700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f9878108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f987e836700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9878108f60 con 0x7f9878108570 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f9877fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f9878108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f9877fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f9878108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40400/0 (socket says 172.21.15.37:40400) 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f9877fff700 1 -- 172.21.15.37:0/802150567 learned_addr learned my addr 172.21.15.37:0/802150567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f9877fff700 1 -- 172.21.15.37:0/802150567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9878109720 con 0x7f9878108570 2024-04-23T18:11:14.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f9877fff700 1 --2- 172.21.15.37:0/802150567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f9878108990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98680098a0 tx=0x7f9868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffb16907b213c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.582+0000 7f9876ffd700 1 -- 172.21.15.37:0/802150567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868004160 con 0x7f9878108570 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f9876ffd700 1 -- 172.21.15.37:0/802150567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9868004300 con 0x7f9878108570 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f9876ffd700 1 -- 172.21.15.37:0/802150567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868011af0 con 0x7f9878108570 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 -- 172.21.15.37:0/802150567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 msgr2=0x7f9878108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 --2- 172.21.15.37:0/802150567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f9878108990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98680098a0 tx=0x7f9868009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 -- 172.21.15.37:0/802150567 shutdown_connections 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 --2- 172.21.15.37:0/802150567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f9878108990 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 -- 172.21.15.37:0/802150567 >> 172.21.15.37:0/802150567 conn(0x7f9878103b70 msgr2=0x7f9878105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 -- 172.21.15.37:0/802150567 shutdown_connections 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.583+0000 7f987e836700 1 -- 172.21.15.37:0/802150567 wait complete. 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f987e836700 1 Processor -- start 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f987e836700 1 -- start start 2024-04-23T18:11:14.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f987e836700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f987819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f987e836700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f987810cad0 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f9877fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f987819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f9877fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f987819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40404/0 (socket says 172.21.15.37:40404) 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.584+0000 7f9877fff700 1 -- 172.21.15.37:0/2396061202 learned_addr learned my addr 172.21.15.37:0/2396061202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f9877fff700 1 -- 172.21.15.37:0/2396061202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987819ca10 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f9877fff700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f987819c4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9868009fb0 tx=0x7f9868011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98680036a0 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f986801a430 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987819cca0 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868019550 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.585+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987819f990 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.586+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f9868019800 con 0x7f9878108570 2024-04-23T18:11:14.795 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.586+0000 7f98757fa700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.586+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 --> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f986003c180 con 0x7f98600395b0 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.586+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f986804d870 con 0x7f9878108570 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.586+0000 7f98777fe700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 msgr2=0x7f986003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.586+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.787+0000 7f98777fe700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 msgr2=0x7f986003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:08.787+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:09.188+0000 7f98777fe700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 msgr2=0x7f986003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:09.188+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:09.989+0000 7f98777fe700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 msgr2=0x7f986003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:09.989+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:11.590+0000 7f98777fe700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 msgr2=0x7f986003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1641631132 2024-04-23T18:11:14.796 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:11.590+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:13.317+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f9868019b40 con 0x7f9878108570 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:13.317+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 msgr2=0x7f986003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:13.317+0000 7f98757fa700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.320+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f9868018350 con 0x7f9878108570 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.320+0000 7f98757fa700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f986003cea0 0x7f986003f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.320+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f986003c180 con 0x7f986003cea0 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.320+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f986003cea0 0x7f986003f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.321+0000 7f98777fe700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f986003cea0 0x7f986003f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f986c003a10 tx=0x7f986c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.322+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f986003c180 con 0x7f986003cea0 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.328+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9878069ba0 con 0x7f986003cea0 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f98757fa700 1 -- 172.21.15.37:0/2396061202 <== mgr.14118 v2:172.21.15.37:6800/4000629980 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9878069ba0 con 0x7f986003cea0 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f986003cea0 msgr2=0x7f986003f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f986003cea0 0x7f986003f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f986c003a10 tx=0x7f986c0092b0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 msgr2=0x7f987819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:14.797 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f987819c4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9868009fb0 tx=0x7f9868011940 comp rx=0 tx=0).stop 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 shutdown_connections 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9878108570 0x7f987819c4d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f986003cea0 0x7f986003f290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 --2- 172.21.15.37:0/2396061202 >> [v2:172.21.15.37:6800/1641631132,v1:172.21.15.37:6801/1641631132] conn(0x7f98600395b0 0x7f986003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 >> 172.21.15.37:0/2396061202 conn(0x7f9878103b70 msgr2=0x7f9878104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 shutdown_connections 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:14.329+0000 7f987e836700 1 -- 172.21.15.37:0/2396061202 wait complete. 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-04-23T18:11:14.798 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-04-23T18:11:15.444 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:11:15.444 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:11:15.444 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:14] ENGINE Bus STARTING 2024-04-23T18:11:15.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:14] ENGINE Serving on https://172.21.15.37:7150 2024-04-23T18:11:15.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:15] ENGINE Serving on http://172.21.15.37:8765 2024-04-23T18:11:15.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:15] ENGINE Bus STARTED 2024-04-23T18:11:15.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:15 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:15.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa05c014700 1 Processor -- start 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa05c014700 1 -- start start 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa05c014700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa05c014700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa05407c830 con 0x7fa05407de40 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa059db0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa059db0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53506/0 (socket says 172.21.15.37:53506) 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.211+0000 7fa059db0700 1 -- 172.21.15.37:0/2169296808 learned_addr learned my addr 172.21.15.37:0/2169296808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa059db0700 1 -- 172.21.15.37:0/2169296808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05407c9b0 con 0x7fa05407de40 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa059db0700 1 --2- 172.21.15.37:0/2169296808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05407c2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098a0 tx=0x7fa050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf48005e633d9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa058dae700 1 -- 172.21.15.37:0/2169296808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050004160 con 0x7fa05407de40 2024-04-23T18:11:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa058dae700 1 -- 172.21.15.37:0/2169296808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa050004300 con 0x7fa05407de40 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa058dae700 1 -- 172.21.15.37:0/2169296808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050011af0 con 0x7fa05407de40 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa05c014700 1 -- 172.21.15.37:0/2169296808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 msgr2=0x7fa05407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.212+0000 7fa05c014700 1 --2- 172.21.15.37:0/2169296808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05407c2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098a0 tx=0x7fa050009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 -- 172.21.15.37:0/2169296808 shutdown_connections 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 --2- 172.21.15.37:0/2169296808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05407c2f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 -- 172.21.15.37:0/2169296808 >> 172.21.15.37:0/2169296808 conn(0x7fa054103b40 msgr2=0x7fa054105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 -- 172.21.15.37:0/2169296808 shutdown_connections 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 -- 172.21.15.37:0/2169296808 wait complete. 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 Processor -- start 2024-04-23T18:11:15.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 -- start start 2024-04-23T18:11:15.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:15.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.213+0000 7fa05c014700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410ead0 con 0x7fa05407de40 2024-04-23T18:11:15.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa059db0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:15.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa059db0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53520/0 (socket says 172.21.15.37:53520) 2024-04-23T18:11:15.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa059db0700 1 -- 172.21.15.37:0/4272298273 learned_addr learned my addr 172.21.15.37:0/4272298273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa059db0700 1 -- 172.21.15.37:0/4272298273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05419ec10 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa059db0700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05419e6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa050009fb0 tx=0x7fa050011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0500036a0 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa05001a430 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05419eea0 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050019550 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.214+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a1b90 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.215+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fa050019850 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.215+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0540640a0 con 0x7fa05407de40 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.215+0000 7fa04affd700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa040039430 0x7fa04003b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:15.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.215+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa05004d720 con 0x7fa05407de40 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.215+0000 7fa0595af700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa040039430 0x7fa04003b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.216+0000 7fa0595af700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa040039430 0x7fa04003b8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa044006fd0 tx=0x7fa044006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.221+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa05001a690 con 0x7fa05407de40 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.377+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa050023390 con 0x7fa05407de40 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.450+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa054104dc0 con 0x7fa040039430 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.455+0000 7fa04affd700 1 -- 172.21.15.37:0/4272298273 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa054104dc0 con 0x7fa040039430 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa040039430 msgr2=0x7fa04003b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa040039430 0x7fa04003b8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa044006fd0 tx=0x7fa044006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 msgr2=0x7fa05419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05419e6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa050009fb0 tx=0x7fa050011940 comp rx=0 tx=0).stop 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 shutdown_connections 2024-04-23T18:11:15.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa040039430 0x7fa04003b8f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:15.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 --2- 172.21.15.37:0/4272298273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05407de40 0x7fa05419e6d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:15.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.457+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 >> 172.21.15.37:0/4272298273 conn(0x7fa054103b40 msgr2=0x7fa0541046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:15.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.458+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 shutdown_connections 2024-04-23T18:11:15.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:15.458+0000 7fa05c014700 1 -- 172.21.15.37:0/4272298273 wait complete. 2024-04-23T18:11:16.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:16 smithi037 ceph-mon[29404]: mgrmap e8: x(active, since 2s) 2024-04-23T18:11:16.539 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:16 smithi037 ceph-mon[29404]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:16.540 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:16 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:16.540 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:16 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.327+0000 7feb08bf6700 1 Processor -- start 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.327+0000 7feb08bf6700 1 -- start start 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.327+0000 7feb08bf6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.327+0000 7feb08bf6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb04063d60 con 0x7feb04108fb0 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0410b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53522/0 (socket says 172.21.15.37:53522) 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0259c700 1 -- 172.21.15.37:0/222835294 learned_addr learned my addr 172.21.15.37:0/222835294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0259c700 1 -- 172.21.15.37:0/222835294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0410b8e0 con 0x7feb04108fb0 2024-04-23T18:11:17.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0259c700 1 --2- 172.21.15.37:0/222835294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0410b3a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feaf80098a0 tx=0x7feaf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8bebf9b8348cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0159a700 1 -- 172.21.15.37:0/222835294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf8004160 con 0x7feb04108fb0 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0159a700 1 -- 172.21.15.37:0/222835294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feaf8004300 con 0x7feb04108fb0 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.328+0000 7feb0159a700 1 -- 172.21.15.37:0/222835294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf8011af0 con 0x7feb04108fb0 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 -- 172.21.15.37:0/222835294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 msgr2=0x7feb0410b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 --2- 172.21.15.37:0/222835294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0410b3a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feaf80098a0 tx=0x7feaf8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 -- 172.21.15.37:0/222835294 shutdown_connections 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 --2- 172.21.15.37:0/222835294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0410b3a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 -- 172.21.15.37:0/222835294 >> 172.21.15.37:0/222835294 conn(0x7feb04102a40 msgr2=0x7feb04104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 -- 172.21.15.37:0/222835294 shutdown_connections 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.329+0000 7feb08bf6700 1 -- 172.21.15.37:0/222835294 wait complete. 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb08bf6700 1 Processor -- start 2024-04-23T18:11:17.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb08bf6700 1 -- start start 2024-04-23T18:11:17.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb08bf6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:17.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb08bf6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410c8b0 con 0x7feb04108fb0 2024-04-23T18:11:17.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb0259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:17.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb0259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53532/0 (socket says 172.21.15.37:53532) 2024-04-23T18:11:17.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb0259c700 1 -- 172.21.15.37:0/3453334762 learned_addr learned my addr 172.21.15.37:0/3453334762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:17.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.330+0000 7feb0259c700 1 -- 172.21.15.37:0/3453334762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0419c990 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.331+0000 7feb0259c700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0419c450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feaf8011ac0 tx=0x7feaf8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.331+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf8003b20 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.331+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feaf801a8c0 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.331+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0419cc20 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.331+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf80248d0 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.331+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0419f910 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.332+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7feaf8022020 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.332+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04069ba0 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.332+0000 7feaf37fe700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7feaec039600 0x7feaec03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.332+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feaf804d890 con 0x7feb04108fb0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.332+0000 7feb01d9b700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7feaec039600 0x7feaec03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:17.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.332+0000 7feb01d9b700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7feaec039600 0x7feaec03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006fd0 tx=0x7feaf4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.337+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feaf800b5a0 con 0x7feb04108fb0 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.545+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7feb04103c20 con 0x7feaec039600 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.546+0000 7feaf37fe700 1 -- 172.21.15.37:0/3453334762 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7feb04103c20 con 0x7feaec039600 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7feaec039600 msgr2=0x7feaec03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7feaec039600 0x7feaec03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006fd0 tx=0x7feaf4006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 msgr2=0x7feb0419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0419c450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feaf8011ac0 tx=0x7feaf8011ba0 comp rx=0 tx=0).stop 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 shutdown_connections 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7feaec039600 0x7feaec03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 --2- 172.21.15.37:0/3453334762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb04108fb0 0x7feb0419c450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 >> 172.21.15.37:0/3453334762 conn(0x7feb04102a40 msgr2=0x7feb04103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 shutdown_connections 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:16.548+0000 7feb08bf6700 1 -- 172.21.15.37:0/3453334762 wait complete. 2024-04-23T18:11:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-04-23T18:11:17.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:17 smithi037 ceph-mon[29404]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: Generating public/private rsa key pair. 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: Your identification has been saved in /tmp/tmpo_gflnf1/key. 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: Your public key has been saved in /tmp/tmpo_gflnf1/key.pub. 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: The key fingerprint is: 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: SHA256:uTVrautlSYcwj/Q9N8b6aEXb0xh2E0lYPzAKEAYJ/ss ceph-af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: The key's randomart image is: 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: +---[RSA 3072]----+ 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | ...o+o. o+o.| 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | . .. . ..oo.| 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | . + . .o| 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | . . B o .+.o| 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | . S B +o=*o| 2024-04-23T18:11:18.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | . . + = =+oo| 2024-04-23T18:11:18.373 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | E . * .. .| 2024-04-23T18:11:18.373 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | .= .o | 2024-04-23T18:11:18.373 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: | o+. .. . | 2024-04-23T18:11:18.373 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: +----[SHA256]-----+ 2024-04-23T18:11:18.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.575+0000 7f10d0872700 1 Processor -- start 2024-04-23T18:11:18.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.575+0000 7f10d0872700 1 -- start start 2024-04-23T18:11:18.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.575+0000 7f10d0872700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.575+0000 7f10d0872700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc108f60 con 0x7f10cc108570 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10ca59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10ca59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53540/0 (socket says 172.21.15.37:53540) 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10ca59c700 1 -- 172.21.15.37:0/2010087821 learned_addr learned my addr 172.21.15.37:0/2010087821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10ca59c700 1 -- 172.21.15.37:0/2010087821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc109720 con 0x7f10cc108570 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10ca59c700 1 --2- 172.21.15.37:0/2010087821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc108990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009e80 tx=0x7f10c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686401fbd9edf885 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10c959a700 1 -- 172.21.15.37:0/2010087821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c4017070 con 0x7f10cc108570 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10c959a700 1 -- 172.21.15.37:0/2010087821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f10c400b8c0 con 0x7f10cc108570 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.576+0000 7f10c959a700 1 -- 172.21.15.37:0/2010087821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c4004490 con 0x7f10cc108570 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 -- 172.21.15.37:0/2010087821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 msgr2=0x7f10cc108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 --2- 172.21.15.37:0/2010087821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc108990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009e80 tx=0x7f10c400b330 comp rx=0 tx=0).stop 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 -- 172.21.15.37:0/2010087821 shutdown_connections 2024-04-23T18:11:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 --2- 172.21.15.37:0/2010087821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc108990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 -- 172.21.15.37:0/2010087821 >> 172.21.15.37:0/2010087821 conn(0x7f10cc103b70 msgr2=0x7f10cc105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 -- 172.21.15.37:0/2010087821 shutdown_connections 2024-04-23T18:11:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.577+0000 7f10d0872700 1 -- 172.21.15.37:0/2010087821 wait complete. 2024-04-23T18:11:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10d0872700 1 Processor -- start 2024-04-23T18:11:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10d0872700 1 -- start start 2024-04-23T18:11:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10d0872700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10d0872700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc10cad0 con 0x7f10cc108570 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10ca59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10ca59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53548/0 (socket says 172.21.15.37:53548) 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.578+0000 7f10ca59c700 1 -- 172.21.15.37:0/2251728450 learned_addr learned my addr 172.21.15.37:0/2251728450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10ca59c700 1 -- 172.21.15.37:0/2251728450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc19c9b0 con 0x7f10cc108570 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10ca59c700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc19c470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10c4004a60 tx=0x7f10c400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c4020020 con 0x7f10cc108570 2024-04-23T18:11:18.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f10c4012430 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc19cc40 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c4017040 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.579+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc19f930 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.580+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f10c4023020 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.580+0000 7f10c37fe700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f10b8039600 0x7f10b803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.580+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc0640a0 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.580+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10c400f050 con 0x7f10cc108570 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.580+0000 7f10c9d9b700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f10b8039600 0x7f10b803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.581+0000 7f10c9d9b700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f10b8039600 0x7f10b803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006fd0 tx=0x7f10b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:18.603 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.586+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10c402a020 con 0x7f10cc108570 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:17.794+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f10cc104d50 con 0x7f10b8039600 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.075+0000 7f10c37fe700 1 -- 172.21.15.37:0/2251728450 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10cc104d50 con 0x7f10b8039600 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f10b8039600 msgr2=0x7f10b803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f10b8039600 0x7f10b803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006fd0 tx=0x7f10b4006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 msgr2=0x7f10cc19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc19c470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10c4004a60 tx=0x7f10c400bd50 comp rx=0 tx=0).stop 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 shutdown_connections 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f10b8039600 0x7f10b803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 --2- 172.21.15.37:0/2251728450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10cc108570 0x7f10cc19c470 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 >> 172.21.15.37:0/2251728450 conn(0x7f10cc103b70 msgr2=0x7f10cc104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 shutdown_connections 2024-04-23T18:11:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:18.077+0000 7f10d0872700 1 -- 172.21.15.37:0/2251728450 wait complete. 2024-04-23T18:11:19.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:19 smithi037 ceph-mon[29404]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:19.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:19 smithi037 ceph-mon[29404]: Generating ssh key... 2024-04-23T18:11:19.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:19 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:19.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:19 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6ba510700 1 Processor -- start 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6ba510700 1 -- start start 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6ba510700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b4108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6ba510700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4108f50 con 0x7fc6b4108560 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b4108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b4108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53556/0 (socket says 172.21.15.37:53556) 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6b3fff700 1 -- 172.21.15.37:0/1405871332 learned_addr learned my addr 172.21.15.37:0/1405871332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.022+0000 7fc6b3fff700 1 -- 172.21.15.37:0/1405871332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b4109710 con 0x7fc6b4108560 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6b3fff700 1 --2- 172.21.15.37:0/1405871332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b4108980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40098a0 tx=0x7fc6a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ed0538ca32c85e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6b2ffd700 1 -- 172.21.15.37:0/1405871332 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4004160 con 0x7fc6b4108560 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6b2ffd700 1 -- 172.21.15.37:0/1405871332 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc6a4004300 con 0x7fc6b4108560 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6b2ffd700 1 -- 172.21.15.37:0/1405871332 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4011af0 con 0x7fc6b4108560 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6ba510700 1 -- 172.21.15.37:0/1405871332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 msgr2=0x7fc6b4108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6ba510700 1 --2- 172.21.15.37:0/1405871332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b4108980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40098a0 tx=0x7fc6a4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6ba510700 1 -- 172.21.15.37:0/1405871332 shutdown_connections 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6ba510700 1 --2- 172.21.15.37:0/1405871332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b4108980 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.023+0000 7fc6ba510700 1 -- 172.21.15.37:0/1405871332 >> 172.21.15.37:0/1405871332 conn(0x7fc6b4103b40 msgr2=0x7fc6b4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.024+0000 7fc6ba510700 1 -- 172.21.15.37:0/1405871332 shutdown_connections 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.024+0000 7fc6ba510700 1 -- 172.21.15.37:0/1405871332 wait complete. 2024-04-23T18:11:19.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.024+0000 7fc6ba510700 1 Processor -- start 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.024+0000 7fc6ba510700 1 -- start start 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.024+0000 7fc6ba510700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.024+0000 7fc6ba510700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410cac0 con 0x7fc6b4108560 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53568/0 (socket says 172.21.15.37:53568) 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b3fff700 1 -- 172.21.15.37:0/4105347592 learned_addr learned my addr 172.21.15.37:0/4105347592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b3fff700 1 -- 172.21.15.37:0/4105347592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b419ec90 con 0x7fc6b4108560 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b3fff700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b419e750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40117b0 tx=0x7fc6a4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4003af0 con 0x7fc6b4108560 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc6a4020070 con 0x7fc6b4108560 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b419ef20 con 0x7fc6b4108560 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4019630 con 0x7fc6b4108560 2024-04-23T18:11:19.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.025+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b41a1c10 con 0x7fc6b4108560 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.026+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc6a401a9c0 con 0x7fc6b4108560 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.026+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b40640a0 con 0x7fc6b4108560 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.026+0000 7fc6b17fa700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fc69c0395b0 0x7fc69c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.026+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc6a404db40 con 0x7fc6b4108560 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.027+0000 7fc6b37fe700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fc69c0395b0 0x7fc69c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.027+0000 7fc6b37fe700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fc69c0395b0 0x7fc69c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8006fd0 tx=0x7fc6a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.032+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6a400e260 con 0x7fc6b4108560 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.242+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc6b4104d50 con 0x7fc69c0395b0 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.242+0000 7fc6b17fa700 1 -- 172.21.15.37:0/4105347592 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc6b4104d50 con 0x7fc69c0395b0 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fc69c0395b0 msgr2=0x7fc69c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fc69c0395b0 0x7fc69c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8006fd0 tx=0x7fc6a8006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 msgr2=0x7fc6b419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b419e750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40117b0 tx=0x7fc6a4011b70 comp rx=0 tx=0).stop 2024-04-23T18:11:19.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 shutdown_connections 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fc69c0395b0 0x7fc69c03ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 --2- 172.21.15.37:0/4105347592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6b4108560 0x7fc6b419e750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 >> 172.21.15.37:0/4105347592 conn(0x7fc6b4103b40 msgr2=0x7fc6b4104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 shutdown_connections 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:19.245+0000 7fc6ba510700 1 -- 172.21.15.37:0/4105347592 wait complete. 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T18:11:19.786 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-04-23T18:11:20.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:20 smithi037 ceph-mon[29404]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:21 smithi037 ceph-mon[29404]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:21 smithi037 ceph-mon[29404]: Deploying cephadm binary to smithi037 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.165+0000 7f8f9347c700 1 Processor -- start 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.165+0000 7f8f9347c700 1 -- start start 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f9347c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c10b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f9347c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c063d40 con 0x7f8f8c108fd0 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f91218700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c10b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:25.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f91218700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c10b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53582/0 (socket says 172.21.15.37:53582) 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f91218700 1 -- 172.21.15.37:0/1794574340 learned_addr learned my addr 172.21.15.37:0/1794574340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f91218700 1 -- 172.21.15.37:0/1794574340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c10b900 con 0x7f8f8c108fd0 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f91218700 1 --2- 172.21.15.37:0/1794574340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c10b3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098a0 tx=0x7f8f88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded8910fbcd893b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.166+0000 7f8f83fff700 1 -- 172.21.15.37:0/1794574340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f88004160 con 0x7f8f8c108fd0 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f83fff700 1 -- 172.21.15.37:0/1794574340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f88004300 con 0x7f8f8c108fd0 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f83fff700 1 -- 172.21.15.37:0/1794574340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f88011af0 con 0x7f8f8c108fd0 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 -- 172.21.15.37:0/1794574340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 msgr2=0x7f8f8c10b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 --2- 172.21.15.37:0/1794574340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c10b3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098a0 tx=0x7f8f88009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 -- 172.21.15.37:0/1794574340 shutdown_connections 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 --2- 172.21.15.37:0/1794574340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c10b3c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 -- 172.21.15.37:0/1794574340 >> 172.21.15.37:0/1794574340 conn(0x7f8f8c102a40 msgr2=0x7f8f8c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 -- 172.21.15.37:0/1794574340 shutdown_connections 2024-04-23T18:11:25.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.167+0000 7f8f9347c700 1 -- 172.21.15.37:0/1794574340 wait complete. 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f9347c700 1 Processor -- start 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f9347c700 1 -- start start 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f9347c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f9347c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10c850 con 0x7f8f8c108fd0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f91218700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f91218700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53592/0 (socket says 172.21.15.37:53592) 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f91218700 1 -- 172.21.15.37:0/268208938 learned_addr learned my addr 172.21.15.37:0/268208938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.168+0000 7f8f91218700 1 -- 172.21.15.37:0/268208938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c19c930 con 0x7f8f8c108fd0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.169+0000 7f8f91218700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c19c3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8f88003f40 tx=0x7f8f88011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.169+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f880038b0 con 0x7f8f8c108fd0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.169+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f88003a50 con 0x7f8f8c108fd0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.169+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c19cc20 con 0x7f8f8c108fd0 2024-04-23T18:11:25.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.169+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f8801a5c0 con 0x7f8f8c108fd0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.169+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c19f8b0 con 0x7f8f8c108fd0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.170+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8f88024430 con 0x7f8f8c108fd0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.170+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c0640a0 con 0x7f8f8c108fd0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.170+0000 7f8f827fc700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f8f78039560 0x7f8f7803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.170+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f8804dae0 con 0x7f8f8c108fd0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.170+0000 7f8f90a17700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f8f78039560 0x7f8f7803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.171+0000 7f8f90a17700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f8f78039560 0x7f8f7803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c006fd0 tx=0x7f8f7c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.175+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f8804d470 con 0x7f8f8c108fd0 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:20.389+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7f8f8c103bf0 con 0x7f8f78039560 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.618+0000 7f8f827fc700 1 -- 172.21.15.37:0/268208938 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8f8c103bf0 con 0x7f8f78039560 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f8f78039560 msgr2=0x7f8f7803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f8f78039560 0x7f8f7803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c006fd0 tx=0x7f8f7c006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:25.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 msgr2=0x7f8f8c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c19c3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8f88003f40 tx=0x7f8f88011890 comp rx=0 tx=0).stop 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 shutdown_connections 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f8f78039560 0x7f8f7803ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 --2- 172.21.15.37:0/268208938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f8c108fd0 0x7f8f8c19c3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 >> 172.21.15.37:0/268208938 conn(0x7f8f8c102a40 msgr2=0x7f8f8c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 shutdown_connections 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:24.621+0000 7f8f9347c700 1 -- 172.21.15.37:0/268208938 wait complete. 2024-04-23T18:11:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-04-23T18:11:25.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:25 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:25.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:25 smithi037 ceph-mon[29404]: Added host smithi037 2024-04-23T18:11:25.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:25 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:26.715 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T18:11:26.715 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2ec39700 1 Processor -- start 2024-04-23T18:11:26.715 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2ec39700 1 -- start start 2024-04-23T18:11:26.715 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2ec39700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc28107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:26.715 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2ec39700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28107e30 con 0x7fcc28107440 2024-04-23T18:11:26.715 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2c9d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc28107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2c9d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc28107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57472/0 (socket says 172.21.15.37:57472) 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.757+0000 7fcc2c9d5700 1 -- 172.21.15.37:0/3515778077 learned_addr learned my addr 172.21.15.37:0/3515778077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.758+0000 7fcc2c9d5700 1 -- 172.21.15.37:0/3515778077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc28108640 con 0x7fcc28107440 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.758+0000 7fcc2c9d5700 1 --2- 172.21.15.37:0/3515778077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc28107860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098a0 tx=0x7fcc1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14baf2cda849aed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.758+0000 7fcc277fe700 1 -- 172.21.15.37:0/3515778077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c004160 con 0x7fcc28107440 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.758+0000 7fcc277fe700 1 -- 172.21.15.37:0/3515778077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc1c004300 con 0x7fcc28107440 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.758+0000 7fcc277fe700 1 -- 172.21.15.37:0/3515778077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c00fb60 con 0x7fcc28107440 2024-04-23T18:11:26.716 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 -- 172.21.15.37:0/3515778077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 msgr2=0x7fcc28107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 --2- 172.21.15.37:0/3515778077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc28107860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098a0 tx=0x7fcc1c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 -- 172.21.15.37:0/3515778077 shutdown_connections 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 --2- 172.21.15.37:0/3515778077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc28107860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 -- 172.21.15.37:0/3515778077 >> 172.21.15.37:0/3515778077 conn(0x7fcc28102a40 msgr2=0x7fcc28104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 -- 172.21.15.37:0/3515778077 shutdown_connections 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.759+0000 7fcc2ec39700 1 -- 172.21.15.37:0/3515778077 wait complete. 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2ec39700 1 Processor -- start 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2ec39700 1 -- start start 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2ec39700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc2819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2ec39700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28062e80 con 0x7fcc28107440 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2c9d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc2819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2c9d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc2819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57476/0 (socket says 172.21.15.37:57476) 2024-04-23T18:11:26.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.760+0000 7fcc2c9d5700 1 -- 172.21.15.37:0/1321557250 learned_addr learned my addr 172.21.15.37:0/1321557250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc2c9d5700 1 -- 172.21.15.37:0/1321557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2819a7c0 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc2c9d5700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc2819a280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00f820 tx=0x7fcc1c0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c027070 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc1c003a20 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc1c022690 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2819aa50 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.761+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2819d740 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.762+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc280640a0 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.763+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcc1c02a070 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.763+0000 7fcc25ffb700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fcc180395b0 0x7fcc1803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.763+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc1c01f070 con 0x7fcc28107440 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.763+0000 7fcc27fff700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fcc180395b0 0x7fcc1803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.764+0000 7fcc27fff700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fcc180395b0 0x7fcc1803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:26.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.768+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc1c015070 con 0x7fcc28107440 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:25.997+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcc28075460 con 0x7fcc180395b0 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.002+0000 7fcc25ffb700 1 -- 172.21.15.37:0/1321557250 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcc28075460 con 0x7fcc180395b0 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fcc180395b0 msgr2=0x7fcc1803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fcc180395b0 0x7fcc1803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 msgr2=0x7fcc2819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc2819a280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00f820 tx=0x7fcc1c0041d0 comp rx=0 tx=0).stop 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 shutdown_connections 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fcc180395b0 0x7fcc1803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 --2- 172.21.15.37:0/1321557250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc28107440 0x7fcc2819a280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 >> 172.21.15.37:0/1321557250 conn(0x7fcc28102a40 msgr2=0x7fcc281908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.004+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 shutdown_connections 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:26.005+0000 7fcc2ec39700 1 -- 172.21.15.37:0/1321557250 wait complete. 2024-04-23T18:11:26.719 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-04-23T18:11:27.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:27 smithi037 ceph-mon[29404]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:27.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:27 smithi037 ceph-mon[29404]: Saving service mon spec with placement count:5 2024-04-23T18:11:27.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:27 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:28.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T18:11:28.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f16787ed700 1 Processor -- start 2024-04-23T18:11:28.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f16787ed700 1 -- start start 2024-04-23T18:11:28.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f16787ed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f1670108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:28.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f16787ed700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1670108f50 con 0x7f1670108560 2024-04-23T18:11:28.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f1676589700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f1670108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f1676589700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f1670108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57484/0 (socket says 172.21.15.37:57484) 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f1676589700 1 -- 172.21.15.37:0/3000053985 learned_addr learned my addr 172.21.15.37:0/3000053985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.139+0000 7f1676589700 1 -- 172.21.15.37:0/3000053985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1670109710 con 0x7f1670108560 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.140+0000 7f1676589700 1 --2- 172.21.15.37:0/3000053985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f1670108980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f166c0098a0 tx=0x7f166c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b16144c00dca6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.140+0000 7f1675587700 1 -- 172.21.15.37:0/3000053985 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f166c004160 con 0x7f1670108560 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.140+0000 7f1675587700 1 -- 172.21.15.37:0/3000053985 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f166c004300 con 0x7f1670108560 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.140+0000 7f1675587700 1 -- 172.21.15.37:0/3000053985 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f166c011af0 con 0x7f1670108560 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.140+0000 7f16787ed700 1 -- 172.21.15.37:0/3000053985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 msgr2=0x7f1670108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.140+0000 7f16787ed700 1 --2- 172.21.15.37:0/3000053985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f1670108980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f166c0098a0 tx=0x7f166c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 -- 172.21.15.37:0/3000053985 shutdown_connections 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 --2- 172.21.15.37:0/3000053985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f1670108980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 -- 172.21.15.37:0/3000053985 >> 172.21.15.37:0/3000053985 conn(0x7f1670103b40 msgr2=0x7f1670105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 -- 172.21.15.37:0/3000053985 shutdown_connections 2024-04-23T18:11:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 -- 172.21.15.37:0/3000053985 wait complete. 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 Processor -- start 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 -- start start 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f167007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.141+0000 7f16787ed700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f167010cac0 con 0x7f1670108560 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f1676589700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f167007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f1676589700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f167007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57500/0 (socket says 172.21.15.37:57500) 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f1676589700 1 -- 172.21.15.37:0/3617029902 learned_addr learned my addr 172.21.15.37:0/3617029902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f1676589700 1 -- 172.21.15.37:0/3617029902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167007e0c0 con 0x7f1670108560 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f1676589700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f167007db80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f166c0117b0 tx=0x7f166c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f166c003af0 con 0x7f1670108560 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167007c2b0 con 0x7f1670108560 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f166c020070 con 0x7f1670108560 2024-04-23T18:11:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f166c019630 con 0x7f1670108560 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.142+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167007c7f0 con 0x7f1670108560 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.143+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f166c0198a0 con 0x7f1670108560 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.143+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16700640a0 con 0x7f1670108560 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.143+0000 7f16677fe700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f165c039600 0x7f165c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.144+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f166c04db30 con 0x7f1670108560 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.144+0000 7f1675d88700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f165c039600 0x7f165c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.144+0000 7f1675d88700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f165c039600 0x7f165c03bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1660006fd0 tx=0x7f1660006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.149+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f166c00e260 con 0x7f1670108560 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.378+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f167007d050 con 0x7f165c039600 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.382+0000 7f16677fe700 1 -- 172.21.15.37:0/3617029902 <== mgr.14118 v2:172.21.15.37:6800/4000629980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f167007d050 con 0x7f165c039600 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.384+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f165c039600 msgr2=0x7f165c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.384+0000 7f16787ed700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f165c039600 0x7f165c03bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1660006fd0 tx=0x7f1660006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.384+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 msgr2=0x7f167007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:28.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.384+0000 7f16787ed700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f167007db80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f166c0117b0 tx=0x7f166c011b70 comp rx=0 tx=0).stop 2024-04-23T18:11:28.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.385+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 shutdown_connections 2024-04-23T18:11:28.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.385+0000 7f16787ed700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f165c039600 0x7f165c03bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:28.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.385+0000 7f16787ed700 1 --2- 172.21.15.37:0/3617029902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1670108560 0x7f167007db80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:28.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.385+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 >> 172.21.15.37:0/3617029902 conn(0x7f1670103b40 msgr2=0x7f1670104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:28.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.385+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 shutdown_connections 2024-04-23T18:11:28.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:27.385+0000 7f16787ed700 1 -- 172.21.15.37:0/3617029902 wait complete. 2024-04-23T18:11:28.478 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:28 smithi037 ceph-mon[29404]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:28.478 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:28 smithi037 ceph-mon[29404]: Saving service mgr spec with placement count:2 2024-04-23T18:11:28.478 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:28 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.825+0000 7f1783e98700 1 Processor -- start 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.825+0000 7f1783e98700 1 -- start start 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.825+0000 7f1783e98700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c107240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1783e98700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f177c063ce0 con 0x7f177c106e20 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1781c34700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1781c34700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57508/0 (socket says 172.21.15.37:57508) 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1781c34700 1 -- 172.21.15.37:0/608758641 learned_addr learned my addr 172.21.15.37:0/608758641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1781c34700 1 -- 172.21.15.37:0/608758641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c107780 con 0x7f177c106e20 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1781c34700 1 --2- 172.21.15.37:0/608758641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c107240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f17780098a0 tx=0x7f1778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304b955c2347f56e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:29.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1780c32700 1 -- 172.21.15.37:0/608758641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1778004160 con 0x7f177c106e20 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1780c32700 1 -- 172.21.15.37:0/608758641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1778004300 con 0x7f177c106e20 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.826+0000 7f1780c32700 1 -- 172.21.15.37:0/608758641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1778011af0 con 0x7f177c106e20 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.827+0000 7f1783e98700 1 -- 172.21.15.37:0/608758641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 msgr2=0x7f177c107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.827+0000 7f1783e98700 1 --2- 172.21.15.37:0/608758641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c107240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f17780098a0 tx=0x7f1778009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.827+0000 7f1783e98700 1 -- 172.21.15.37:0/608758641 shutdown_connections 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.827+0000 7f1783e98700 1 --2- 172.21.15.37:0/608758641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c107240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.827+0000 7f1783e98700 1 -- 172.21.15.37:0/608758641 >> 172.21.15.37:0/608758641 conn(0x7f177c102a40 msgr2=0x7f177c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.828+0000 7f1783e98700 1 -- 172.21.15.37:0/608758641 shutdown_connections 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.828+0000 7f1783e98700 1 -- 172.21.15.37:0/608758641 wait complete. 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.828+0000 7f1783e98700 1 Processor -- start 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.828+0000 7f1783e98700 1 -- start start 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1783e98700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:29.851 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1783e98700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1087b0 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1781c34700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1781c34700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57520/0 (socket says 172.21.15.37:57520) 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1781c34700 1 -- 172.21.15.37:0/3351373560 learned_addr learned my addr 172.21.15.37:0/3351373560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1781c34700 1 -- 172.21.15.37:0/3351373560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c19eb70 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.829+0000 7f1781c34700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c19e630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1778011ac0 tx=0x7f1778011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.830+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1778003b20 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.830+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f177801a8c0 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.830+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c19ee00 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.830+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17780248d0 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.830+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1a1af0 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.831+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c0640a0 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.831+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1778022020 con 0x7f177c106e20 2024-04-23T18:11:29.852 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.831+0000 7f1772ffd700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f1768039600 0x7f176803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.831+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f177804d810 con 0x7f177c106e20 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.831+0000 7f1781433700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f1768039600 0x7f176803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.832+0000 7f1781433700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f1768039600 0x7f176803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f176c006fd0 tx=0x7f176c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:28.836+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f177801ab20 con 0x7f177c106e20 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.046+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f177c0008d0 con 0x7f177c106e20 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.049+0000 7f1772ffd700 1 -- 172.21.15.37:0/3351373560 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f177802d3f0 con 0x7f177c106e20 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f1768039600 msgr2=0x7f176803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f1768039600 0x7f176803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f176c006fd0 tx=0x7f176c006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 msgr2=0x7f177c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c19e630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1778011ac0 tx=0x7f1778011ba0 comp rx=0 tx=0).stop 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 shutdown_connections 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f1768039600 0x7f176803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:29.853 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 --2- 172.21.15.37:0/3351373560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c106e20 0x7f177c19e630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:29.854 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 >> 172.21.15.37:0/3351373560 conn(0x7f177c102a40 msgr2=0x7f177c195290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:29.854 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 shutdown_connections 2024-04-23T18:11:29.854 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:29.051+0000 7f1783e98700 1 -- 172.21.15.37:0/3351373560 wait complete. 2024-04-23T18:11:30.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:30 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3351373560' entity='client.admin' 2024-04-23T18:11:30.270 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:30 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:30.270 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:30 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:31.110 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.299+0000 7fa6c9211700 1 Processor -- start 2024-04-23T18:11:31.110 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.299+0000 7fa6c9211700 1 -- start start 2024-04-23T18:11:31.110 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.299+0000 7fa6c9211700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c4108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:31.110 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.299+0000 7fa6c9211700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4108f60 con 0x7fa6c4108570 2024-04-23T18:11:31.111 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.299+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c4108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:31.111 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c4108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57524/0 (socket says 172.21.15.37:57524) 2024-04-23T18:11:31.111 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c2d9d700 1 -- 172.21.15.37:0/1702413570 learned_addr learned my addr 172.21.15.37:0/1702413570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:31.111 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c2d9d700 1 -- 172.21.15.37:0/1702413570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c4109720 con 0x7fa6c4108570 2024-04-23T18:11:31.111 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c2d9d700 1 --2- 172.21.15.37:0/1702413570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c4108990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098a0 tx=0x7fa6b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e3c8ab9788704a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:31.111 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c1d9b700 1 -- 172.21.15.37:0/1702413570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b8004160 con 0x7fa6c4108570 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c1d9b700 1 -- 172.21.15.37:0/1702413570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8004300 con 0x7fa6c4108570 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.300+0000 7fa6c1d9b700 1 -- 172.21.15.37:0/1702413570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b8011a20 con 0x7fa6c4108570 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 -- 172.21.15.37:0/1702413570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 msgr2=0x7fa6c4108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 --2- 172.21.15.37:0/1702413570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c4108990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098a0 tx=0x7fa6b8009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 -- 172.21.15.37:0/1702413570 shutdown_connections 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 --2- 172.21.15.37:0/1702413570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c4108990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 -- 172.21.15.37:0/1702413570 >> 172.21.15.37:0/1702413570 conn(0x7fa6c4103b70 msgr2=0x7fa6c4105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:31.112 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 -- 172.21.15.37:0/1702413570 shutdown_connections 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.301+0000 7fa6c9211700 1 -- 172.21.15.37:0/1702413570 wait complete. 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c9211700 1 Processor -- start 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c9211700 1 -- start start 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c9211700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c407db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c9211700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c410cad0 con 0x7fa6c4108570 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c407db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c407db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57526/0 (socket says 172.21.15.37:57526) 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.302+0000 7fa6c2d9d700 1 -- 172.21.15.37:0/3297278223 learned_addr learned my addr 172.21.15.37:0/3297278223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6c2d9d700 1 -- 172.21.15.37:0/3297278223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c407e0c0 con 0x7fa6c4108570 2024-04-23T18:11:31.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6c2d9d700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c407db80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80119f0 tx=0x7fa6b8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b80039e0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c407c2b0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b801b8f0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b80237a0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.303+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c407c7f0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.304+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b0005360 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.304+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa6b802c3f0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.304+0000 7fa6abfff700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.304+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6b804ddf0 con 0x7fa6c4108570 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.304+0000 7fa6c259c700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.305+0000 7fa6c259c700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4006fd0 tx=0x7fa6b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:31.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.309+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6b801f030 con 0x7fa6c4108570 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.538+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa6b0005650 con 0x7fa6c4108570 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.541+0000 7fa6abfff700 1 -- 172.21.15.37:0/3297278223 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa6b80292a0 con 0x7fa6c4108570 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa6ac0395b0 msgr2=0x7fa6ac03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4006fd0 tx=0x7fa6b4006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 msgr2=0x7fa6c407db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c407db80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80119f0 tx=0x7fa6b8011ad0 comp rx=0 tx=0).stop 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 shutdown_connections 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 --2- 172.21.15.37:0/3297278223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6c4108570 0x7fa6c407db80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 >> 172.21.15.37:0/3297278223 conn(0x7fa6c4103b70 msgr2=0x7fa6c41046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 shutdown_connections 2024-04-23T18:11:31.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:30.544+0000 7fa6c9211700 1 -- 172.21.15.37:0/3297278223 wait complete. 2024-04-23T18:11:31.116 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-04-23T18:11:31.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:31 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3297278223' entity='client.admin' 2024-04-23T18:11:32.811 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:32 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3210228627' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T18:11:32.812 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:32 smithi037 ceph-mon[29404]: from='mgr.14118 172.21.15.37:0/2060125792' entity='mgr.x' 2024-04-23T18:11:32.812 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:32 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setuser ceph since I am not root 2024-04-23T18:11:32.812 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:32 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setgroup ceph since I am not root 2024-04-23T18:11:32.812 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:32 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:32.732+0000 7f8328488200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:11:33.070 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:32 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:32.958+0000 7f8328488200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:11:33.071 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:33 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:33.076+0000 7f8328488200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.574+0000 7fa381098700 1 Processor -- start 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.574+0000 7fa381098700 1 -- start start 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa381098700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c10b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa381098700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c063d60 con 0x7fa37c108fb0 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c10b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c10b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57536/0 (socket says 172.21.15.37:57536) 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37ad9d700 1 -- 172.21.15.37:0/2176656828 learned_addr learned my addr 172.21.15.37:0/2176656828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37ad9d700 1 -- 172.21.15.37:0/2176656828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c10b8e0 con 0x7fa37c108fb0 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37ad9d700 1 --2- 172.21.15.37:0/2176656828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c10b3a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0098a0 tx=0x7fa36c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9379be9cedbdd333 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:33.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37a59c700 1 -- 172.21.15.37:0/2176656828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa36c004160 con 0x7fa37c108fb0 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37a59c700 1 -- 172.21.15.37:0/2176656828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa36c004300 con 0x7fa37c108fb0 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.575+0000 7fa37a59c700 1 -- 172.21.15.37:0/2176656828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa36c011a20 con 0x7fa37c108fb0 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 -- 172.21.15.37:0/2176656828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 msgr2=0x7fa37c10b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 --2- 172.21.15.37:0/2176656828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c10b3a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0098a0 tx=0x7fa36c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 -- 172.21.15.37:0/2176656828 shutdown_connections 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 --2- 172.21.15.37:0/2176656828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c10b3a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 -- 172.21.15.37:0/2176656828 >> 172.21.15.37:0/2176656828 conn(0x7fa37c102a40 msgr2=0x7fa37c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 -- 172.21.15.37:0/2176656828 shutdown_connections 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.576+0000 7fa381098700 1 -- 172.21.15.37:0/2176656828 wait complete. 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa381098700 1 Processor -- start 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa381098700 1 -- start start 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa381098700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa381098700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c10c8f0 con 0x7fa37c108fb0 2024-04-23T18:11:33.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa37ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa37ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57544/0 (socket says 172.21.15.37:57544) 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa37ad9d700 1 -- 172.21.15.37:0/3210228627 learned_addr learned my addr 172.21.15.37:0/3210228627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.577+0000 7fa37ad9d700 1 -- 172.21.15.37:0/3210228627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c19c920 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.578+0000 7fa37ad9d700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c19c3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0116e0 tx=0x7fa36c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.578+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa36c003a60 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.578+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa36c01b8f0 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.578+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c19cbb0 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.578+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa36c019860 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.578+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c19f8a0 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.579+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa36c019b10 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.579+0000 7fa378d99700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa3680395b0 0x7fa36803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.579+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c0640a0 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.579+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa36c04df40 con 0x7fa37c108fb0 2024-04-23T18:11:33.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.579+0000 7fa373fff700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa3680395b0 0x7fa36803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.579+0000 7fa373fff700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa3680395b0 0x7fa36803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa364006fd0 tx=0x7fa364006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.584+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa36c01f030 con 0x7fa37c108fb0 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:31.852+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa37c19fa70 con 0x7fa37c108fb0 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.544+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa36c029260 con 0x7fa37c108fb0 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.546+0000 7fa378d99700 1 -- 172.21.15.37:0/3210228627 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa36c01a750 con 0x7fa37c108fb0 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa3680395b0 msgr2=0x7fa36803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa3680395b0 0x7fa36803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa364006fd0 tx=0x7fa364006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 msgr2=0x7fa37c19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c19c3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0116e0 tx=0x7fa36c011aa0 comp rx=0 tx=0).stop 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 shutdown_connections 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7fa3680395b0 0x7fa36803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 --2- 172.21.15.37:0/3210228627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa37c108fb0 0x7fa37c19c3e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 >> 172.21.15.37:0/3210228627 conn(0x7fa37c102a40 msgr2=0x7fa37c103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:33.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 shutdown_connections 2024-04-23T18:11:33.203 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:32.548+0000 7fa381098700 1 -- 172.21.15.37:0/3210228627 wait complete. 2024-04-23T18:11:33.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:33 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3210228627' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T18:11:33.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:33 smithi037 ceph-mon[29404]: mgrmap e9: x(active, since 19s) 2024-04-23T18:11:33.622 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:33 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:33.402+0000 7f8328488200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:11:33.622 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:33 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:33.493+0000 7f8328488200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:11:34.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:11:34.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T18:11:34.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.646+0000 7f84386eb700 1 Processor -- start 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.646+0000 7f84386eb700 1 -- start start 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.646+0000 7f84386eb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.646+0000 7f84386eb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8430063d60 con 0x7f8430108fb0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8436487700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8436487700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57568/0 (socket says 172.21.15.37:57568) 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8436487700 1 -- 172.21.15.37:0/1448874576 learned_addr learned my addr 172.21.15.37:0/1448874576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8436487700 1 -- 172.21.15.37:0/1448874576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843010b8e0 con 0x7f8430108fb0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8436487700 1 --2- 172.21.15.37:0/1448874576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843010b3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098a0 tx=0x7f842c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ad638ae81098bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8435485700 1 -- 172.21.15.37:0/1448874576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c004160 con 0x7f8430108fb0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8435485700 1 -- 172.21.15.37:0/1448874576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c004300 con 0x7f8430108fb0 2024-04-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.647+0000 7f8435485700 1 -- 172.21.15.37:0/1448874576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c011a20 con 0x7f8430108fb0 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 -- 172.21.15.37:0/1448874576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 msgr2=0x7f843010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 --2- 172.21.15.37:0/1448874576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843010b3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098a0 tx=0x7f842c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 -- 172.21.15.37:0/1448874576 shutdown_connections 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 --2- 172.21.15.37:0/1448874576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843010b3a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 -- 172.21.15.37:0/1448874576 >> 172.21.15.37:0/1448874576 conn(0x7f8430102a40 msgr2=0x7f8430104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 -- 172.21.15.37:0/1448874576 shutdown_connections 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.648+0000 7f84386eb700 1 -- 172.21.15.37:0/1448874576 wait complete. 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f84386eb700 1 Processor -- start 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f84386eb700 1 -- start start 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f84386eb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f84386eb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f843010c8f0 con 0x7f8430108fb0 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f8436487700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f8436487700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57574/0 (socket says 172.21.15.37:57574) 2024-04-23T18:11:34.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f8436487700 1 -- 172.21.15.37:0/2711453517 learned_addr learned my addr 172.21.15.37:0/2711453517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.649+0000 7f8436487700 1 -- 172.21.15.37:0/2711453517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843019c920 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.650+0000 7f8436487700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843019c3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.650+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c003aa0 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.650+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843019cbb0 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.650+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c01b8f0 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.650+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843019f8a0 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.650+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842c023870 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.651+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f842c023b70 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.651+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84300640a0 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.651+0000 7f84237fe700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 0x7f841c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.651+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f842c04dfc0 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.651+0000 7f8435c86700 1 -- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 msgr2=0x7f841c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.651+0000 7f8435c86700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 0x7f841c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.656+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f842c01f030 con 0x7f8430108fb0 2024-04-23T18:11:34.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.852+0000 7f8435c86700 1 -- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 msgr2=0x7f841c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.852+0000 7f8435c86700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 0x7f841c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.971+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f843019fe90 con 0x7f8430108fb0 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.971+0000 7f84237fe700 1 -- 172.21.15.37:0/2711453517 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f842c04d950 con 0x7f8430108fb0 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 msgr2=0x7f841c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 0x7f841c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 msgr2=0x7f843019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843019c3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c011aa0 comp rx=0 tx=0).stop 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 shutdown_connections 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f841c039600 0x7f841c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 --2- 172.21.15.37:0/2711453517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8430108fb0 0x7f843019c3e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 >> 172.21.15.37:0/2711453517 conn(0x7f8430102a40 msgr2=0x7f8430103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.973+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 shutdown_connections 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:33.974+0000 7f84386eb700 1 -- 172.21.15.37:0/2711453517 wait complete. 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-23T18:11:34.484 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-04-23T18:11:34.849 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2711453517' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:11:34.849 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:34 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:34.762+0000 7f8328488200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:11:35.116 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:34 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:34.945+0000 7f8328488200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:11:35.116 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:35.122+0000 7f8328488200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:11:35.724 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:35.423+0000 7f8328488200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:11:35.725 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:35.520+0000 7f8328488200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:11:35.725 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:35.730+0000 7f8328488200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:11:36.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:35.932+0000 7f8328488200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:11:36.871 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:36 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:36.558+0000 7f8328488200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:11:37.302 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:36 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:36.997+0000 7f8328488200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:11:37.303 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:37 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:37.098+0000 7f8328488200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:11:37.303 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:37 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:37.308+0000 7f8328488200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:11:38.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:37 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:37.852+0000 7f8328488200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:11:38.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:37 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:37.949+0000 7f8328488200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:11:38.561 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:38.203+0000 7f8328488200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:11:38.561 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:38.343+0000 7f8328488200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:11:38.561 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:38.444+0000 7f8328488200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:11:38.561 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:38.566+0000 7f8328488200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:11:39.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:38.906+0000 7f8328488200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:11:39.824 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:39.543+0000 7f8328488200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:11:39.824 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:11:39.777+0000 7f8328488200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:11:40.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: Active manager daemon x restarted 2024-04-23T18:11:40.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: Activating manager daemon x 2024-04-23T18:11:40.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: mgrmap e10: x(active, starting, since 0.00349433s) 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: Manager daemon x is now available 2024-04-23T18:11:40.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:41.007 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:11:41.007 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:11:41.007 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:40 smithi037 ceph-mon[29404]: mgrmap e11: x(active, since 1.00607s) 2024-04-23T18:11:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.883+0000 7f311093e700 1 Processor -- start 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f311093e700 1 -- start start 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f311093e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c107250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f311093e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f310c07be40 con 0x7f310c106e30 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f310a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c107250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f310a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c107250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52234/0 (socket says 172.21.15.37:52234) 2024-04-23T18:11:41.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f310a59c700 1 -- 172.21.15.37:0/1601321404 learned_addr learned my addr 172.21.15.37:0/1601321404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.884+0000 7f310a59c700 1 -- 172.21.15.37:0/1601321404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c107790 con 0x7f310c106e30 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.885+0000 7f310a59c700 1 --2- 172.21.15.37:0/1601321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c107250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0098a0 tx=0x7f30fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed383ca6596825f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.885+0000 7f310959a700 1 -- 172.21.15.37:0/1601321404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc004160 con 0x7f310c106e30 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.885+0000 7f310959a700 1 -- 172.21.15.37:0/1601321404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc004300 con 0x7f310c106e30 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.885+0000 7f310959a700 1 -- 172.21.15.37:0/1601321404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc011a20 con 0x7f310c106e30 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.885+0000 7f311093e700 1 -- 172.21.15.37:0/1601321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 msgr2=0x7f310c107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.885+0000 7f311093e700 1 --2- 172.21.15.37:0/1601321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c107250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0098a0 tx=0x7f30fc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 -- 172.21.15.37:0/1601321404 shutdown_connections 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 --2- 172.21.15.37:0/1601321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c107250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 -- 172.21.15.37:0/1601321404 >> 172.21.15.37:0/1601321404 conn(0x7f310c102a50 msgr2=0x7f310c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:41.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 -- 172.21.15.37:0/1601321404 shutdown_connections 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 -- 172.21.15.37:0/1601321404 wait complete. 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 Processor -- start 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.886+0000 7f311093e700 1 -- start start 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f311093e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f311093e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f310c108800 con 0x7f310c106e30 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f310a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f310a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52250/0 (socket says 172.21.15.37:52250) 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f310a59c700 1 -- 172.21.15.37:0/205223041 learned_addr learned my addr 172.21.15.37:0/205223041 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f310a59c700 1 -- 172.21.15.37:0/205223041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c19eb50 con 0x7f310c106e30 2024-04-23T18:11:41.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f310a59c700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c19e610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30fc004660 tx=0x7f30fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc003aa0 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c19ede0 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc01b8f0 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc023900 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.887+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c1a1ad0 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.888+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f30fc01baa0 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.889+0000 7f30fb7fe700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.889+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 --> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f30f403c230 con 0x7f30f4039660 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.889+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f30fc04de30 con 0x7f310c106e30 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.889+0000 7f3109d9b700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 msgr2=0x7f30f403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:34.889+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:35.090+0000 7f3109d9b700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 msgr2=0x7f30f403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:35.090+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:11:41.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:35.491+0000 7f3109d9b700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 msgr2=0x7f30f403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:35.491+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:36.292+0000 7f3109d9b700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 msgr2=0x7f30f403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:36.292+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:37.893+0000 7f3109d9b700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 msgr2=0x7f30f403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4000629980 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:37.893+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:39.781+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f30fc023b70 con 0x7f310c106e30 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:39.781+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 msgr2=0x7f30f403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:39.781+0000 7f30fb7fe700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.782+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f30fc04ff20 con 0x7f310c106e30 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.782+0000 7f30fb7fe700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f30f403cff0 0x7f30f403f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.782+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f30fc046f50 con 0x7f30f403cff0 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.783+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f30f403cff0 0x7f30f403f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:41.612 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.783+0000 7f3109d9b700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f30f403cff0 0x7f30f403f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3100003a10 tx=0x7f3100009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.784+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f30fc046f50 con 0x7f30f403cff0 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f310c069ba0 con 0x7f30f403cff0 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f30fb7fe700 1 -- 172.21.15.37:0/205223041 <== mgr.14150 v2:172.21.15.37:6800/3159991179 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f310c069ba0 con 0x7f30f403cff0 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f30f403cff0 msgr2=0x7f30f403f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f30f403cff0 0x7f30f403f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3100003a10 tx=0x7f3100009270 comp rx=0 tx=0).stop 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 msgr2=0x7f310c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c19e610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30fc004660 tx=0x7f30fc011aa0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 shutdown_connections 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f30f403cff0 0x7f30f403f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c106e30 0x7f310c19e610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 --2- 172.21.15.37:0/205223041 >> [v2:172.21.15.37:6800/4000629980,v1:172.21.15.37:6801/4000629980] conn(0x7f30f4039660 0x7f30f403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 >> 172.21.15.37:0/205223041 conn(0x7f310c102a50 msgr2=0x7f310c195250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:41.613 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.796+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 shutdown_connections 2024-04-23T18:11:41.614 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:40.797+0000 7f311093e700 1 -- 172.21.15.37:0/205223041 wait complete. 2024-04-23T18:11:41.614 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-04-23T18:11:41.614 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-04-23T18:11:41.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:41 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:41] ENGINE Bus STARTING 2024-04-23T18:11:41.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.059+0000 7f3189d0c700 1 Processor -- start 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.059+0000 7f3189d0c700 1 -- start start 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.060+0000 7f3189d0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f3184107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.060+0000 7f3189d0c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3184108370 con 0x7f3184107980 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.060+0000 7f31837fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f3184107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.060+0000 7f31837fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f3184107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52324/0 (socket says 172.21.15.37:52324) 2024-04-23T18:11:42.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.060+0000 7f31837fe700 1 -- 172.21.15.37:0/2348824468 learned_addr learned my addr 172.21.15.37:0/2348824468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.060+0000 7f31837fe700 1 -- 172.21.15.37:0/2348824468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31841086a0 con 0x7f3184107980 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f31837fe700 1 --2- 172.21.15.37:0/2348824468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f3184107da0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f316c0098a0 tx=0x7f316c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5583477d65894117 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f31827fc700 1 -- 172.21.15.37:0/2348824468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c004160 con 0x7f3184107980 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f31827fc700 1 -- 172.21.15.37:0/2348824468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c004300 con 0x7f3184107980 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f31827fc700 1 -- 172.21.15.37:0/2348824468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c011a20 con 0x7f3184107980 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f3189d0c700 1 -- 172.21.15.37:0/2348824468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 msgr2=0x7f3184107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f3189d0c700 1 --2- 172.21.15.37:0/2348824468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f3184107da0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f316c0098a0 tx=0x7f316c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f3189d0c700 1 -- 172.21.15.37:0/2348824468 shutdown_connections 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f3189d0c700 1 --2- 172.21.15.37:0/2348824468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f3184107da0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.061+0000 7f3189d0c700 1 -- 172.21.15.37:0/2348824468 >> 172.21.15.37:0/2348824468 conn(0x7f31841021a0 msgr2=0x7f31841045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.062+0000 7f3189d0c700 1 -- 172.21.15.37:0/2348824468 shutdown_connections 2024-04-23T18:11:42.943 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.062+0000 7f3189d0c700 1 -- 172.21.15.37:0/2348824468 wait complete. 2024-04-23T18:11:42.944 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.062+0000 7f3189d0c700 1 Processor -- start 2024-04-23T18:11:42.944 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.062+0000 7f3189d0c700 1 -- start start 2024-04-23T18:11:42.944 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.062+0000 7f3189d0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f318419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:42.944 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.062+0000 7f3189d0c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f318410ba50 con 0x7f3184107980 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f31837fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f318419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f31837fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f318419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52336/0 (socket says 172.21.15.37:52336) 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f31837fe700 1 -- 172.21.15.37:0/1124694625 learned_addr learned my addr 172.21.15.37:0/1124694625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f31837fe700 1 -- 172.21.15.37:0/1124694625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318419ebe0 con 0x7f3184107980 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f31837fe700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f318419e6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f316c0116e0 tx=0x7f316c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c0039e0 con 0x7f3184107980 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c01b8f0 con 0x7f3184107980 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c023760 con 0x7f3184107980 2024-04-23T18:11:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318419ee70 con 0x7f3184107980 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.063+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31841a1b10 con 0x7f3184107980 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.064+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31840640a0 con 0x7f3184107980 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.065+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f316c0239d0 con 0x7f3184107980 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.065+0000 7f3180ff9700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f31700394a0 0x7f317003b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.065+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f316c04dff0 con 0x7f3184107980 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.065+0000 7f3182ffd700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f31700394a0 0x7f317003b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.065+0000 7f3182ffd700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f31700394a0 0x7f317003b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3174006fd0 tx=0x7f3174006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.070+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f316c01f030 con 0x7f3184107980 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.330+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f318402e240 con 0x7f31700394a0 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.403+0000 7f3180ff9700 1 -- 172.21.15.37:0/1124694625 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f318402e240 con 0x7f31700394a0 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f31700394a0 msgr2=0x7f317003b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f31700394a0 0x7f317003b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3174006fd0 tx=0x7f3174006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 msgr2=0x7f318419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:42.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f318419e6a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f316c0116e0 tx=0x7f316c011aa0 comp rx=0 tx=0).stop 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 shutdown_connections 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f31700394a0 0x7f317003b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 --2- 172.21.15.37:0/1124694625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3184107980 0x7f318419e6a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.405+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 >> 172.21.15.37:0/1124694625 conn(0x7f31841021a0 msgr2=0x7f3184102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.406+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 shutdown_connections 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:42.406+0000 7f3189d0c700 1 -- 172.21.15.37:0/1124694625 wait complete. 2024-04-23T18:11:42.949 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-04-23T18:11:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:41] ENGINE Serving on https://172.21.15.37:7150 2024-04-23T18:11:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:41] ENGINE Serving on http://172.21.15.37:8765 2024-04-23T18:11:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: [23/Apr/2024:18:11:41] ENGINE Bus STARTED 2024-04-23T18:11:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:43.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:43.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:43.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: mgrmap e12: x(active, since 2s) 2024-04-23T18:11:43.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:44.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fo28SmZIS18kLufJW2PPIO44WAAoS.xGjwTJWLj6UJNTE5MOK6iOK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713895903, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.308+0000 7fb8f73bf700 1 Processor -- start 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f73bf700 1 -- start start 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f73bf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f0108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f73bf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0108f50 con 0x7fb8f0108560 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f515b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f0108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f515b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f0108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52340/0 (socket says 172.21.15.37:52340) 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f515b700 1 -- 172.21.15.37:0/1431074057 learned_addr learned my addr 172.21.15.37:0/1431074057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.309+0000 7fb8f515b700 1 -- 172.21.15.37:0/1431074057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f0109710 con 0x7fb8f0108560 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8f515b700 1 --2- 172.21.15.37:0/1431074057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f0108980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0098a0 tx=0x7fb8ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707e8e57c76de913 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:44.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8e7fff700 1 -- 172.21.15.37:0/1431074057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec004160 con 0x7fb8f0108560 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8e7fff700 1 -- 172.21.15.37:0/1431074057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ec004300 con 0x7fb8f0108560 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8e7fff700 1 -- 172.21.15.37:0/1431074057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec011a20 con 0x7fb8f0108560 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8f73bf700 1 -- 172.21.15.37:0/1431074057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 msgr2=0x7fb8f0108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8f73bf700 1 --2- 172.21.15.37:0/1431074057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f0108980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0098a0 tx=0x7fb8ec009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8f73bf700 1 -- 172.21.15.37:0/1431074057 shutdown_connections 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8f73bf700 1 --2- 172.21.15.37:0/1431074057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f0108980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.310+0000 7fb8f73bf700 1 -- 172.21.15.37:0/1431074057 >> 172.21.15.37:0/1431074057 conn(0x7fb8f0103b40 msgr2=0x7fb8f0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.311+0000 7fb8f73bf700 1 -- 172.21.15.37:0/1431074057 shutdown_connections 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.311+0000 7fb8f73bf700 1 -- 172.21.15.37:0/1431074057 wait complete. 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.311+0000 7fb8f73bf700 1 Processor -- start 2024-04-23T18:11:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.311+0000 7fb8f73bf700 1 -- start start 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.311+0000 7fb8f73bf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.311+0000 7fb8f73bf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010cac0 con 0x7fb8f0108560 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f515b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f515b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52352/0 (socket says 172.21.15.37:52352) 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f515b700 1 -- 172.21.15.37:0/3511567604 learned_addr learned my addr 172.21.15.37:0/3511567604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f515b700 1 -- 172.21.15.37:0/3511567604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f007e0c0 con 0x7fb8f0108560 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f515b700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f007db80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0119f0 tx=0x7fb8ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec0039e0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ec01b8f0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f007c2b0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec0237a0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.312+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f007c7f0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.313+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb8ec02c3f0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.313+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f00640a0 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.313+0000 7fb8e67fc700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb8dc039610 0x7fb8dc03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.313+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb8ec04e120 con 0x7fb8f0108560 2024-04-23T18:11:44.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.313+0000 7fb8f495a700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb8dc039610 0x7fb8dc03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.314+0000 7fb8f495a700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb8dc039610 0x7fb8dc03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0006fd0 tx=0x7fb8e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.318+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8ec01f030 con 0x7fb8f0108560 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.544+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb8f007ca70 con 0x7fb8dc039610 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.791+0000 7fb8e67fc700 1 -- 172.21.15.37:0/3511567604 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb8f007ca70 con 0x7fb8dc039610 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.793+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb8dc039610 msgr2=0x7fb8dc03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.793+0000 7fb8f73bf700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb8dc039610 0x7fb8dc03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0006fd0 tx=0x7fb8e0006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.793+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 msgr2=0x7fb8f007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.793+0000 7fb8f73bf700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f007db80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0119f0 tx=0x7fb8ec011ad0 comp rx=0 tx=0).stop 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.794+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 shutdown_connections 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.794+0000 7fb8f73bf700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb8dc039610 0x7fb8dc03bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:44.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.794+0000 7fb8f73bf700 1 --2- 172.21.15.37:0/3511567604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8f0108560 0x7fb8f007db80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:44.390 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.794+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 >> 172.21.15.37:0/3511567604 conn(0x7fb8f0103b40 msgr2=0x7fb8f01046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:44.390 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.794+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 shutdown_connections 2024-04-23T18:11:44.390 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:43.794+0000 7fb8f73bf700 1 -- 172.21.15.37:0/3511567604 wait complete. 2024-04-23T18:11:44.390 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-04-23T18:11:44.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:44 smithi037 ceph-mon[29404]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:44.704 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:45.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T18:11:45.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.817+0000 7fe5bc398700 1 Processor -- start 2024-04-23T18:11:45.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.817+0000 7fe5bc398700 1 -- start start 2024-04-23T18:11:45.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.817+0000 7fe5bc398700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:45.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.817+0000 7fe5bc398700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b407c8e0 con 0x7fe5b407de60 2024-04-23T18:11:45.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5ba134700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5ba134700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34806/0 (socket says 172.21.15.37:34806) 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5ba134700 1 -- 172.21.15.37:0/3433867905 learned_addr learned my addr 172.21.15.37:0/3433867905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5ba134700 1 -- 172.21.15.37:0/3433867905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b407ca60 con 0x7fe5b407de60 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5ba134700 1 --2- 172.21.15.37:0/3433867905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b407c310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009e80 tx=0x7fe5a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02d228065b73115 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5b9132700 1 -- 172.21.15.37:0/3433867905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5a4017070 con 0x7fe5b407de60 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5b9132700 1 -- 172.21.15.37:0/3433867905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a400b8c0 con 0x7fe5b407de60 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.818+0000 7fe5b9132700 1 -- 172.21.15.37:0/3433867905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5a4004490 con 0x7fe5b407de60 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 -- 172.21.15.37:0/3433867905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 msgr2=0x7fe5b407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 --2- 172.21.15.37:0/3433867905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b407c310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009e80 tx=0x7fe5a400b330 comp rx=0 tx=0).stop 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 -- 172.21.15.37:0/3433867905 shutdown_connections 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 --2- 172.21.15.37:0/3433867905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b407c310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 -- 172.21.15.37:0/3433867905 >> 172.21.15.37:0/3433867905 conn(0x7fe5b4103b40 msgr2=0x7fe5b4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 -- 172.21.15.37:0/3433867905 shutdown_connections 2024-04-23T18:11:45.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.819+0000 7fe5bc398700 1 -- 172.21.15.37:0/3433867905 wait complete. 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5bc398700 1 Processor -- start 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5bc398700 1 -- start start 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5bc398700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5bc398700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b410eb00 con 0x7fe5b407de60 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5ba134700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5ba134700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34812/0 (socket says 172.21.15.37:34812) 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.820+0000 7fe5ba134700 1 -- 172.21.15.37:0/1000497147 learned_addr learned my addr 172.21.15.37:0/1000497147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5ba134700 1 -- 172.21.15.37:0/1000497147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b419ebd0 con 0x7fe5b407de60 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5ba134700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b419e690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009e50 tx=0x7fe5a400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5a4020020 con 0x7fe5b407de60 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b419ee60 con 0x7fe5b407de60 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a4003b50 con 0x7fe5b407de60 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5a4017040 con 0x7fe5b407de60 2024-04-23T18:11:45.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.821+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41a1b50 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.822+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe5a4023020 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.822+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b40640a0 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.822+0000 7fe5b37fe700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe59c0395b0 0x7fe59c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.822+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe5a400f050 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.822+0000 7fe5b9933700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe59c0395b0 0x7fe59c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.823+0000 7fe5b9933700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe59c0395b0 0x7fe59c03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006fd0 tx=0x7fe5a8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:44.827+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5a402a020 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.067+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe5b41a2120 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.067+0000 7fe5b37fe700 1 -- 172.21.15.37:0/1000497147 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe5a40102f0 con 0x7fe5b407de60 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe59c0395b0 msgr2=0x7fe59c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe59c0395b0 0x7fe59c03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006fd0 tx=0x7fe5a8006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 msgr2=0x7fe5b419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b419e690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009e50 tx=0x7fe5a400bd50 comp rx=0 tx=0).stop 2024-04-23T18:11:45.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 shutdown_connections 2024-04-23T18:11:45.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe59c0395b0 0x7fe59c03ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:45.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.070+0000 7fe5bc398700 1 --2- 172.21.15.37:0/1000497147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b407de60 0x7fe5b419e690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:45.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.071+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 >> 172.21.15.37:0/1000497147 conn(0x7fe5b4103b40 msgr2=0x7fe5b4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:45.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.071+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 shutdown_connections 2024-04-23T18:11:45.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:45.071+0000 7fe5bc398700 1 -- 172.21.15.37:0/1000497147 wait complete. 2024-04-23T18:11:45.649 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:11:45.649 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-23T18:11:45.656 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:11:45.656 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-23T18:11:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-23T18:11:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T18:11:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-04-23T18:11:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037:8443/ 2024-04-23T18:11:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-04-23T18:11:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: Password: y6dzkifvf1 2024-04-23T18:11:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:45.659 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config directory 2024-04-23T18:11:45.659 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-04-23T18:11:45.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:45 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1000497147' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T18:11:47.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:47 smithi037 ceph-mon[29404]: mgrmap e13: x(active, since 6s) 2024-04-23T18:11:47.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.953+0000 7f4ecf9e2700 1 Processor -- start 2024-04-23T18:11:47.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.953+0000 7f4ecf9e2700 1 -- start start 2024-04-23T18:11:47.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.953+0000 7f4ecf9e2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:47.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.953+0000 7f4ecf9e2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec807c830 con 0x7f4ec807de40 2024-04-23T18:11:47.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.953+0000 7f4ecd77e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:47.717 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ecd77e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34824/0 (socket says 172.21.15.37:34824) 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ecd77e700 1 -- 172.21.15.37:0/4092958116 learned_addr learned my addr 172.21.15.37:0/4092958116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ecd77e700 1 -- 172.21.15.37:0/4092958116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec807c9b0 con 0x7f4ec807de40 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ecd77e700 1 --2- 172.21.15.37:0/4092958116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec807c2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e80 tx=0x7f4eb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bc2393649937a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ec7fff700 1 -- 172.21.15.37:0/4092958116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4eb8017070 con 0x7f4ec807de40 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ec7fff700 1 -- 172.21.15.37:0/4092958116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb800b8c0 con 0x7f4ec807de40 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.954+0000 7f4ec7fff700 1 -- 172.21.15.37:0/4092958116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4eb8004490 con 0x7f4ec807de40 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/4092958116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 msgr2=0x7f4ec807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 --2- 172.21.15.37:0/4092958116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec807c2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009e80 tx=0x7f4eb800b330 comp rx=0 tx=0).stop 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/4092958116 shutdown_connections 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 --2- 172.21.15.37:0/4092958116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec807c2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/4092958116 >> 172.21.15.37:0/4092958116 conn(0x7f4ec8103b10 msgr2=0x7f4ec8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/4092958116 shutdown_connections 2024-04-23T18:11:47.718 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/4092958116 wait complete. 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 Processor -- start 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.955+0000 7f4ecf9e2700 1 -- start start 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecf9e2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecf9e2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec810eab0 con 0x7f4ec807de40 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecd77e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecd77e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34840/0 (socket says 172.21.15.37:34840) 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecd77e700 1 -- 172.21.15.37:0/954765310 learned_addr learned my addr 172.21.15.37:0/954765310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecd77e700 1 -- 172.21.15.37:0/954765310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec819ebb0 con 0x7f4ec807de40 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecd77e700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec819e670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b7b0 tx=0x7f4eb8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4eb8020020 con 0x7f4ec807de40 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb80046a0 con 0x7f4ec807de40 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec819ee40 con 0x7f4ec807de40 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.956+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4eb8017040 con 0x7f4ec807de40 2024-04-23T18:11:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.957+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81a1b30 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.957+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4eb8023020 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.958+0000 7f4ec67fc700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4eb0039700 0x7f4eb003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.958+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec80640a0 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.958+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4eb800f050 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.958+0000 7f4eccf7d700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4eb0039700 0x7f4eb003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.958+0000 7f4eccf7d700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4eb0039700 0x7f4eb003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:46.963+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4eb802a020 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.170+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4ec81a1d00 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.171+0000 7f4ec67fc700 1 -- 172.21.15.37:0/954765310 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4eb80115f0 con 0x7f4ec807de40 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4eb0039700 msgr2=0x7f4eb003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4eb0039700 0x7f4eb003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 msgr2=0x7f4ec819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec819e670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b7b0 tx=0x7f4eb8009390 comp rx=0 tx=0).stop 2024-04-23T18:11:47.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 shutdown_connections 2024-04-23T18:11:47.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4eb0039700 0x7f4eb003bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:47.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 --2- 172.21.15.37:0/954765310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ec807de40 0x7f4ec819e670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:47.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 >> 172.21.15.37:0/954765310 conn(0x7f4ec8103b10 msgr2=0x7f4ec8104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:47.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 shutdown_connections 2024-04-23T18:11:47.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:47.173+0000 7f4ecf9e2700 1 -- 172.21.15.37:0/954765310 wait complete. 2024-04-23T18:11:49.066 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.125+0000 7f1702413700 1 Processor -- start 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f1702413700 1 -- start start 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f1702413700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f1702413700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc07c830 con 0x7f16fc07de40 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f16fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f16fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34856/0 (socket says 172.21.15.37:34856) 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f16fbfff700 1 -- 172.21.15.37:0/3945625387 learned_addr learned my addr 172.21.15.37:0/3945625387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f16fbfff700 1 -- 172.21.15.37:0/3945625387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc07c9b0 con 0x7f16fc07de40 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.126+0000 7f16fbfff700 1 --2- 172.21.15.37:0/3945625387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc07c2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098a0 tx=0x7f16ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d135bec4fd1201 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f16faffd700 1 -- 172.21.15.37:0/3945625387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec004160 con 0x7f16fc07de40 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f16faffd700 1 -- 172.21.15.37:0/3945625387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ec004300 con 0x7f16fc07de40 2024-04-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f16faffd700 1 -- 172.21.15.37:0/3945625387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec011a20 con 0x7f16fc07de40 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 -- 172.21.15.37:0/3945625387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 msgr2=0x7f16fc07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 --2- 172.21.15.37:0/3945625387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc07c2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098a0 tx=0x7f16ec009bb0 comp rx=0 tx=0).stop 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 -- 172.21.15.37:0/3945625387 shutdown_connections 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 --2- 172.21.15.37:0/3945625387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc07c2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 -- 172.21.15.37:0/3945625387 >> 172.21.15.37:0/3945625387 conn(0x7f16fc103b40 msgr2=0x7f16fc105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 -- 172.21.15.37:0/3945625387 shutdown_connections 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.127+0000 7f1702413700 1 -- 172.21.15.37:0/3945625387 wait complete. 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f1702413700 1 Processor -- start 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f1702413700 1 -- start start 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f1702413700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:49.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f1702413700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc10eac0 con 0x7f16fc07de40 2024-04-23T18:11:49.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f16fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:49.072 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f16fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34864/0 (socket says 172.21.15.37:34864) 2024-04-23T18:11:49.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.128+0000 7f16fbfff700 1 -- 172.21.15.37:0/3828139527 learned_addr learned my addr 172.21.15.37:0/3828139527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:49.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f16fbfff700 1 -- 172.21.15.37:0/3828139527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc19eba0 con 0x7f16fc07de40 2024-04-23T18:11:49.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f16fbfff700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc19e660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0116e0 tx=0x7f16ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:49.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec003b20 con 0x7f16fc07de40 2024-04-23T18:11:49.073 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16ec01b8f0 con 0x7f16fc07de40 2024-04-23T18:11:49.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc19ee30 con 0x7f16fc07de40 2024-04-23T18:11:49.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc1a1b20 con 0x7f16fc07de40 2024-04-23T18:11:49.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.129+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec0238c0 con 0x7f16fc07de40 2024-04-23T18:11:49.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.130+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f16ec01baa0 con 0x7f16fc07de40 2024-04-23T18:11:49.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.130+0000 7f16f97fa700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16e40396b0 0x7f16e403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:49.074 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.130+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f16ec04e1e0 con 0x7f16fc07de40 2024-04-23T18:11:49.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.130+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc0640a0 con 0x7f16fc07de40 2024-04-23T18:11:49.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.130+0000 7f16fb7fe700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16e40396b0 0x7f16e403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:49.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.131+0000 7f16fb7fe700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16e40396b0 0x7f16e403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f16f0006fd0 tx=0x7f16f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:49.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.135+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16ec01f030 con 0x7f16fc07de40 2024-04-23T18:11:49.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.459+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f16fc1a1cf0 con 0x7f16fc07de40 2024-04-23T18:11:49.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.462+0000 7f16f97fa700 1 -- 172.21.15.37:0/3828139527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f16ec02bb90 con 0x7f16fc07de40 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16e40396b0 msgr2=0x7f16e403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16e40396b0 0x7f16e403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f16f0006fd0 tx=0x7f16f0006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 msgr2=0x7f16fc19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc19e660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0116e0 tx=0x7f16ec011aa0 comp rx=0 tx=0).stop 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 shutdown_connections 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16e40396b0 0x7f16e403bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:49.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 --2- 172.21.15.37:0/3828139527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16fc07de40 0x7f16fc19e660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 >> 172.21.15.37:0/3828139527 conn(0x7f16fc103b40 msgr2=0x7f16fc104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 shutdown_connections 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-23T18:11:48.464+0000 7f1702413700 1 -- 172.21.15.37:0/3828139527 wait complete. 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.077 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-04-23T18:11:49.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.079 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T18:11:49.079 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:11:49.079 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-04-23T18:11:49.114 INFO:tasks.cephadm:Fetching config... 2024-04-23T18:11:49.114 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:11:49.114 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T18:11:49.132 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T18:11:49.132 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:11:49.132 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T18:11:49.191 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T18:11:49.191 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:11:49.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-23T18:11:49.256 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T18:11:49.257 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:11:49.257 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T18:11:49.313 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T18:11:49.313 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:11:49.389 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:11:49.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:11:49.440 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:11:49.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:11:49.491 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp8ayYOMBJ/8LrevXIyEBYqMqXb4gU3dtb4EE+wSC+w/KRl6yGLIHs+DAT/dKy2vhfznaFFM1FX7qbyQoUpLDJv+jhmSL8UcIlQWfOHHEp481OzSL/3Frzccz2G3vI/3P5IwlEcfPO2PcUmqqizE8MocXGMvWdiWFU7Lpt/1XC3kXIRNp/W67aufsFonrwO7RjJdf9/6adCeCba94lPSGBoaYC93HFz06WmwVcGgkF5AeVazvgtcvZt5szqeKyJ0+5/b0Pj/d77ZAtoZFu0IgKm3IjlivNg/7t6ARja5gjKiA45xCfL548ODt6Yx+ZAbvHFzwWdhms+AarpldJGnQrUa3rGiHXVcmOgfrfhwYTEG1XiRpXXcZka1m/hji+A1BW3b/kfFx71wkPY9TJqZCvtHTJbeO5IUAPD6pLI+3HeXW+zO/HGLmILaH6l+XLya3qf3O0KKRJ5cuZDC2+I6AKHF4MgxkHeVqBtYjm3QZh0GjxJhivBwjV3MOTRLcyn3M= ceph-af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:11:49.503 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T18:11:49.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:49 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3828139527' entity='client.admin' 2024-04-23T18:11:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:11:51.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 -- 172.21.15.37:0/894168796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 msgr2=0x7f318c1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 --2- 172.21.15.37:0/894168796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1017c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c009c20 comp rx=0 tx=0).stop 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 -- 172.21.15.37:0/894168796 shutdown_connections 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 --2- 172.21.15.37:0/894168796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1017c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 -- 172.21.15.37:0/894168796 >> 172.21.15.37:0/894168796 conn(0x7f318c0fc940 msgr2=0x7f318c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 -- 172.21.15.37:0/894168796 shutdown_connections 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.865+0000 7f31919dc700 1 -- 172.21.15.37:0/894168796 wait complete. 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f31919dc700 1 Processor -- start 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f31919dc700 1 -- start start 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f31919dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f31919dc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f318c1072e0 con 0x7f318c1013a0 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f318affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:51.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f318affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34880/0 (socket says 172.21.15.37:34880) 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f318affd700 1 -- 172.21.15.37:0/3623050359 learned_addr learned my addr 172.21.15.37:0/3623050359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f318affd700 1 -- 172.21.15.37:0/3623050359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c196f20 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.866+0000 7f318affd700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1969e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f317c005800 tx=0x7f317c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.867+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c027070 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.867+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c1971b0 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.867+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317c00be10 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.867+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f317c00f460 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.867+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c199ea0 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.868+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f317c02a070 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.868+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c107a60 con 0x7f318c1013a0 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.868+0000 7f31909da700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3174039650 0x7f317403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.868+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f317c01e030 con 0x7f318c1013a0 2024-04-23T18:11:51.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.868+0000 7f318a7fc700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3174039650 0x7f317403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:51.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.868+0000 7f318a7fc700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3174039650 0x7f317403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3180006fd0 tx=0x7f3180006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:51.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:51.873+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f317c013070 con 0x7f318c1013a0 2024-04-23T18:11:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.083+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f318c18fd40 con 0x7f318c1013a0 2024-04-23T18:11:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.085+0000 7f31909da700 1 -- 172.21.15.37:0/3623050359 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f317c031070 con 0x7f318c1013a0 2024-04-23T18:11:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3174039650 msgr2=0x7f317403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3174039650 0x7f317403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3180006fd0 tx=0x7f3180006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 msgr2=0x7f318c1969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1969e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f317c005800 tx=0x7f317c009f60 comp rx=0 tx=0).stop 2024-04-23T18:11:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 shutdown_connections 2024-04-23T18:11:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3174039650 0x7f317403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 --2- 172.21.15.37:0/3623050359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f318c1013a0 0x7f318c1969e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 >> 172.21.15.37:0/3623050359 conn(0x7f318c0fc940 msgr2=0x7f318c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 shutdown_connections 2024-04-23T18:11:52.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:52.088+0000 7f31919dc700 1 -- 172.21.15.37:0/3623050359 wait complete. 2024-04-23T18:11:52.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:52.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:52.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:11:52.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:52.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:52.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:52.652 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T18:11:52.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T18:11:52.894 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:11:53.326 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:53 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3623050359' entity='client.admin' 2024-04-23T18:11:53.326 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:53.326 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 -- 172.21.15.37:0/1561675672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 msgr2=0x7f0dd8102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 --2- 172.21.15.37:0/1561675672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8102900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009910 tx=0x7f0dc8009c20 comp rx=0 tx=0).stop 2024-04-23T18:11:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 -- 172.21.15.37:0/1561675672 shutdown_connections 2024-04-23T18:11:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 --2- 172.21.15.37:0/1561675672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8102900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 -- 172.21.15.37:0/1561675672 >> 172.21.15.37:0/1561675672 conn(0x7f0dd80fda40 msgr2=0x7f0dd80ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 -- 172.21.15.37:0/1561675672 shutdown_connections 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.457+0000 7f0ddd754700 1 -- 172.21.15.37:0/1561675672 wait complete. 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0ddd754700 1 Processor -- start 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0ddd754700 1 -- start start 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0ddd754700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0ddd754700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8108420 con 0x7f0dd81024e0 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0dd6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0dd6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34900/0 (socket says 172.21.15.37:34900) 2024-04-23T18:11:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.458+0000 7f0dd6ffd700 1 -- 172.21.15.37:0/3280664438 learned_addr learned my addr 172.21.15.37:0/3280664438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0dd6ffd700 1 -- 172.21.15.37:0/3280664438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd8198070 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0dd6ffd700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8197b30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8005800 tx=0x7f0dc8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc8027070 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd8198300 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc800be10 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc800f460 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.459+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd819afa0 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.460+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0dc802a070 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.460+0000 7f0dbffff700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0dc0039650 0x7f0dc003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.460+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd8108ba0 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.460+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0dc801e030 con 0x7f0dd81024e0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.460+0000 7f0dd67fc700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0dc0039650 0x7f0dc003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.461+0000 7f0dd67fc700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0dc0039650 0x7f0dc003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc006fd0 tx=0x7f0dcc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:54.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.465+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0dc8013070 con 0x7f0dd81024e0 2024-04-23T18:11:54.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.678+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0dd802e0b0 con 0x7f0dc0039650 2024-04-23T18:11:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.681+0000 7f0dbffff700 1 -- 172.21.15.37:0/3280664438 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0dd802e0b0 con 0x7f0dc0039650 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0dc0039650 msgr2=0x7f0dc003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0dc0039650 0x7f0dc003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc006fd0 tx=0x7f0dcc006e40 comp rx=0 tx=0).stop 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 msgr2=0x7f0dd8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8197b30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8005800 tx=0x7f0dc8009f60 comp rx=0 tx=0).stop 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 shutdown_connections 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0dc0039650 0x7f0dc003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 --2- 172.21.15.37:0/3280664438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd81024e0 0x7f0dd8197b30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 >> 172.21.15.37:0/3280664438 conn(0x7f0dd80fda40 msgr2=0x7f0dd80fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 shutdown_connections 2024-04-23T18:11:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:54.683+0000 7f0ddd754700 1 -- 172.21.15.37:0/3280664438 wait complete. 2024-04-23T18:11:55.419 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi172 2024-04-23T18:11:55.419 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:11:55.419 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:11:55.438 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:11:55.438 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:11:55.496 INFO:tasks.cephadm:Adding host smithi172 to orchestrator... 2024-04-23T18:11:55.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch host add smithi172 2024-04-23T18:11:55.745 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:11:55.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:55.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:55.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:11:55.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:11:55.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:11:55.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:11:55.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:11:55.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:11:55.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:11:55.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:55.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:55.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:11:57.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 -- 172.21.15.37:0/4078650213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 msgr2=0x7f4680101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 --2- 172.21.15.37:0/4078650213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f4680101580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f467000b0a0 tx=0x7f467000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 -- 172.21.15.37:0/4078650213 shutdown_connections 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 --2- 172.21.15.37:0/4078650213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f4680101580 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 -- 172.21.15.37:0/4078650213 >> 172.21.15.37:0/4078650213 conn(0x7f46800fce40 msgr2=0x7f46800ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 -- 172.21.15.37:0/4078650213 shutdown_connections 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.139+0000 7f4685861700 1 -- 172.21.15.37:0/4078650213 wait complete. 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f4685861700 1 Processor -- start 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f4685861700 1 -- start start 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f4685861700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f468019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f4685861700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f468010eb90 con 0x7f46801011a0 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f467effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f468019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:57.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f467effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f468019a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53936/0 (socket says 172.21.15.37:53936) 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f467effd700 1 -- 172.21.15.37:0/102842849 learned_addr learned my addr 172.21.15.37:0/102842849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.140+0000 7f467effd700 1 -- 172.21.15.37:0/102842849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468019a5b0 con 0x7f46801011a0 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.141+0000 7f467effd700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f468019a070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f467000d010 tx=0x7f4670003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.141+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4670021020 con 0x7f46801011a0 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.141+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46700096a0 con 0x7f46801011a0 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.141+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468019a840 con 0x7f46801011a0 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.141+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4670017040 con 0x7f46801011a0 2024-04-23T18:11:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.141+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468019e8f0 con 0x7f46801011a0 2024-04-23T18:11:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.142+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4670024020 con 0x7f46801011a0 2024-04-23T18:11:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.142+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468006b190 con 0x7f46801011a0 2024-04-23T18:11:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.142+0000 7f468485f700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f466c0396a0 0x7f466c03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:11:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.142+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f467000f050 con 0x7f46801011a0 2024-04-23T18:11:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.142+0000 7f467e7fc700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f466c0396a0 0x7f466c03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:11:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.142+0000 7f467e7fc700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f466c0396a0 0x7f466c03bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4668006fd0 tx=0x7f4668006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:11:57.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.147+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f467002b020 con 0x7f46801011a0 2024-04-23T18:11:57.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:11:57.360+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}) -- 0x7f468019b380 con 0x7f466c0396a0 2024-04-23T18:11:58.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:57 smithi037 ceph-mon[29404]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:11:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:11:58 smithi037 ceph-mon[29404]: Deploying cephadm binary to smithi172 2024-04-23T18:12:01.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:00 smithi037 ceph-mon[29404]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:01.648 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi172' with addr '172.21.15.172' 2024-04-23T18:12:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.652+0000 7f468485f700 1 -- 172.21.15.37:0/102842849 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f468019b380 con 0x7f466c0396a0 2024-04-23T18:12:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f466c0396a0 msgr2=0x7f466c03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f466c0396a0 0x7f466c03bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4668006fd0 tx=0x7f4668006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 msgr2=0x7f468019a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f468019a070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f467000d010 tx=0x7f4670003ce0 comp rx=0 tx=0).stop 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 shutdown_connections 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f466c0396a0 0x7f466c03bb60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 --2- 172.21.15.37:0/102842849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46801011a0 0x7f468019a070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 >> 172.21.15.37:0/102842849 conn(0x7f46800fce40 msgr2=0x7f46800fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 shutdown_connections 2024-04-23T18:12:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:01.655+0000 7f4685861700 1 -- 172.21.15.37:0/102842849 wait complete. 2024-04-23T18:12:02.243 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:12:02.492 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:02.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:02.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:02 smithi037 ceph-mon[29404]: Added host smithi172 2024-04-23T18:12:02.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:02.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:02 smithi037 ceph-mon[29404]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:02.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:03.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:03 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 -- 172.21.15.37:0/1625221397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 msgr2=0x7f5cb8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 --2- 172.21.15.37:0/1625221397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8108930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009910 tx=0x7f5ca8009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 -- 172.21.15.37:0/1625221397 shutdown_connections 2024-04-23T18:12:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 --2- 172.21.15.37:0/1625221397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8108930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 -- 172.21.15.37:0/1625221397 >> 172.21.15.37:0/1625221397 conn(0x7f5cb80fdf70 msgr2=0x7f5cb8100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 -- 172.21.15.37:0/1625221397 shutdown_connections 2024-04-23T18:12:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.071+0000 7f5cbd192700 1 -- 172.21.15.37:0/1625221397 wait complete. 2024-04-23T18:12:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cbd192700 1 Processor -- start 2024-04-23T18:12:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cbd192700 1 -- start start 2024-04-23T18:12:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cbd192700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cbd192700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb810c9c0 con 0x7f5cb8108550 2024-04-23T18:12:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cb6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cb6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53952/0 (socket says 172.21.15.37:53952) 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.072+0000 7f5cb6d9d700 1 -- 172.21.15.37:0/2162061777 learned_addr learned my addr 172.21.15.37:0/2162061777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5cb6d9d700 1 -- 172.21.15.37:0/2162061777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb8076be0 con 0x7f5cb8108550 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5cb6d9d700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006010 tx=0x7f5ca8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ca8026070 con 0x7f5cb8108550 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca800bc00 con 0x7f5cb8108550 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb8075280 con 0x7f5cb8108550 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ca8003910 con 0x7f5cb8108550 2024-04-23T18:12:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.073+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb8075740 con 0x7f5cb8108550 2024-04-23T18:12:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.074+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5ca8029070 con 0x7f5cb8108550 2024-04-23T18:12:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.074+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb8103bc0 con 0x7f5cb8108550 2024-04-23T18:12:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.074+0000 7f5c9ffff700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5ca00396b0 0x7f5ca003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.074+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5ca801e070 con 0x7f5cb8108550 2024-04-23T18:12:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.074+0000 7f5cb659c700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5ca00396b0 0x7f5ca003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.075+0000 7f5cb659c700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5ca00396b0 0x7f5ca003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006fd0 tx=0x7f5cac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.080+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ca8013070 con 0x7f5cb8108550 2024-04-23T18:12:04.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.295+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cb80762a0 con 0x7f5ca00396b0 2024-04-23T18:12:04.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.296+0000 7f5c9ffff700 1 -- 172.21.15.37:0/2162061777 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f5cb80762a0 con 0x7f5ca00396b0 2024-04-23T18:12:04.290 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:12:04.291 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}] 2024-04-23T18:12:04.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5ca00396b0 msgr2=0x7f5ca003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5ca00396b0 0x7f5ca003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006fd0 tx=0x7f5cac006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 msgr2=0x7f5cb8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006010 tx=0x7f5ca8005780 comp rx=0 tx=0).stop 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 shutdown_connections 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5ca00396b0 0x7f5ca003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 --2- 172.21.15.37:0/2162061777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cb8108550 0x7f5cb8074d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.298+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 >> 172.21.15.37:0/2162061777 conn(0x7f5cb80fdf70 msgr2=0x7f5cb80fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.299+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 shutdown_connections 2024-04-23T18:12:04.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:04.299+0000 7f5cbd192700 1 -- 172.21.15.37:0/2162061777 wait complete. 2024-04-23T18:12:04.888 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-04-23T18:12:04.889 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:12:04.889 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:12:04.907 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:12:04.908 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:04.965 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-04-23T18:12:04.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch host add smithi173 2024-04-23T18:12:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:05.549 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:05 smithi037 ceph-mon[29404]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:05.549 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:05.549 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:05 smithi037 ceph-mon[29404]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 -- 172.21.15.37:0/2820009806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 msgr2=0x7f6ecc103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 --2- 172.21.15.37:0/2820009806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc103a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009910 tx=0x7f6ebc009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 -- 172.21.15.37:0/2820009806 shutdown_connections 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 --2- 172.21.15.37:0/2820009806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc103a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 -- 172.21.15.37:0/2820009806 >> 172.21.15.37:0/2820009806 conn(0x7f6ecc0fcea0 msgr2=0x7f6ecc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 -- 172.21.15.37:0/2820009806 shutdown_connections 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 -- 172.21.15.37:0/2820009806 wait complete. 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 Processor -- start 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.597+0000 7f6ed2283700 1 -- start start 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ed2283700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ed2283700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc10c9d0 con 0x7f6ecc103650 2024-04-23T18:12:06.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ecbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ecbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46000/0 (socket says 172.21.15.37:46000) 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ecbfff700 1 -- 172.21.15.37:0/3793628414 learned_addr learned my addr 172.21.15.37:0/3793628414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ecbfff700 1 -- 172.21.15.37:0/3793628414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc19a680 con 0x7f6ecc103650 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ecbfff700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc19a140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0056d0 tx=0x7f6ebc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.598+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ebc026070 con 0x7f6ecc103650 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.599+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc004370 con 0x7f6ecc103650 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.599+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ebc0039b0 con 0x7f6ecc103650 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.599+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc19a910 con 0x7f6ecc103650 2024-04-23T18:12:06.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.599+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc19ad70 con 0x7f6ecc103650 2024-04-23T18:12:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.599+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6ebc029070 con 0x7f6ecc103650 2024-04-23T18:12:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.600+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc06b190 con 0x7f6ecc103650 2024-04-23T18:12:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.600+0000 7f6ec97fa700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6eb8039650 0x7f6eb803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.600+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ebc01e070 con 0x7f6ecc103650 2024-04-23T18:12:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.600+0000 7f6ecb7fe700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6eb8039650 0x7f6eb803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.600+0000 7f6ecb7fe700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6eb8039650 0x7f6eb803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4006fd0 tx=0x7f6eb4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:06.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.605+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ebc013070 con 0x7f6ecc103650 2024-04-23T18:12:06.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:06.816+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) -- 0x7f6ecc19b4c0 con 0x7f6eb8039650 2024-04-23T18:12:07.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:07 smithi037 ceph-mon[29404]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: Deploying cephadm binary to smithi173 2024-04-23T18:12:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:12:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:08 smithi037 ceph-mon[29404]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:12:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:09 smithi037 ceph-mon[29404]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:09 smithi037 ceph-mon[29404]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:11.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.016+0000 7f6ec97fa700 1 -- 172.21.15.37:0/3793628414 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6ecc19b4c0 con 0x7f6eb8039650 2024-04-23T18:12:11.011 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6eb8039650 msgr2=0x7f6eb803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6eb8039650 0x7f6eb803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4006fd0 tx=0x7f6eb4006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 msgr2=0x7f6ecc19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc19a140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0056d0 tx=0x7f6ebc0057b0 comp rx=0 tx=0).stop 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 shutdown_connections 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6eb8039650 0x7f6eb803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 --2- 172.21.15.37:0/3793628414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ecc103650 0x7f6ecc19a140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 >> 172.21.15.37:0/3793628414 conn(0x7f6ecc0fcea0 msgr2=0x7f6ecc1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 shutdown_connections 2024-04-23T18:12:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:11.018+0000 7f6ed2283700 1 -- 172.21.15.37:0/3793628414 wait complete. 2024-04-23T18:12:11.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:11.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:11.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:11.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:11.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:11.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:11.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:11.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:11.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:12:11.848 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:12.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:12 smithi037 ceph-mon[29404]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:12:12.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:12 smithi037 ceph-mon[29404]: Added host smithi173 2024-04-23T18:12:12.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:12 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.331+0000 7ff1a9740700 1 -- 172.21.15.37:0/1144007339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 msgr2=0x7ff1a4103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.331+0000 7ff1a9740700 1 --2- 172.21.15.37:0/1144007339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a4103a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff194009910 tx=0x7ff194009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 -- 172.21.15.37:0/1144007339 shutdown_connections 2024-04-23T18:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 --2- 172.21.15.37:0/1144007339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a4103a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 -- 172.21.15.37:0/1144007339 >> 172.21.15.37:0/1144007339 conn(0x7ff1a40fce40 msgr2=0x7ff1a40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 -- 172.21.15.37:0/1144007339 shutdown_connections 2024-04-23T18:12:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 -- 172.21.15.37:0/1144007339 wait complete. 2024-04-23T18:12:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 Processor -- start 2024-04-23T18:12:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.332+0000 7ff1a9740700 1 -- start start 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a9740700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a9740700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a410c9b0 con 0x7ff1a41036b0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46024/0 (socket says 172.21.15.37:46024) 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a2ffd700 1 -- 172.21.15.37:0/4251964473 learned_addr learned my addr 172.21.15.37:0/4251964473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a2ffd700 1 -- 172.21.15.37:0/4251964473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a419a6e0 con 0x7ff1a41036b0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a2ffd700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a419a1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff194006010 tx=0x7ff194005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff194026070 con 0x7ff1a41036b0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a419a970 con 0x7ff1a41036b0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19400bc00 con 0x7ff1a41036b0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.333+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff194003910 con 0x7ff1a41036b0 2024-04-23T18:12:13.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.334+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a419add0 con 0x7ff1a41036b0 2024-04-23T18:12:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.334+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff194029070 con 0x7ff1a41036b0 2024-04-23T18:12:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.334+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a406b0f0 con 0x7ff1a41036b0 2024-04-23T18:12:13.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.335+0000 7ff18bfff700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff18c0396b0 0x7ff18c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:13.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.335+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff19401e070 con 0x7ff1a41036b0 2024-04-23T18:12:13.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.335+0000 7ff1a27fc700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff18c0396b0 0x7ff18c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:13.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.335+0000 7ff1a27fc700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff18c0396b0 0x7ff18c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:13.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.340+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff194013070 con 0x7ff1a41036b0 2024-04-23T18:12:13.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.551+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1a419b500 con 0x7ff18c0396b0 2024-04-23T18:12:13.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.553+0000 7ff18bfff700 1 -- 172.21.15.37:0/4251964473 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7ff1a419b500 con 0x7ff18c0396b0 2024-04-23T18:12:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:12:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff18c0396b0 msgr2=0x7ff18c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff18c0396b0 0x7ff18c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 msgr2=0x7ff1a419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a419a1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff194006010 tx=0x7ff194005780 comp rx=0 tx=0).stop 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 shutdown_connections 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff18c0396b0 0x7ff18c03bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 --2- 172.21.15.37:0/4251964473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1a41036b0 0x7ff1a419a1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 >> 172.21.15.37:0/4251964473 conn(0x7ff1a40fce40 msgr2=0x7ff1a4108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 shutdown_connections 2024-04-23T18:12:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:13.555+0000 7ff1a9740700 1 -- 172.21.15.37:0/4251964473 wait complete. 2024-04-23T18:12:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:13 smithi037 ceph-mon[29404]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:14.185 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T18:12:14.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T18:12:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:14.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:14 smithi037 ceph-mon[29404]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:12:14.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:14 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 -- 172.21.15.37:0/3708942709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 msgr2=0x7f3ac8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 --2- 172.21.15.37:0/3708942709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8108930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009910 tx=0x7f3ab8009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 -- 172.21.15.37:0/3708942709 shutdown_connections 2024-04-23T18:12:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 --2- 172.21.15.37:0/3708942709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8108930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 -- 172.21.15.37:0/3708942709 >> 172.21.15.37:0/3708942709 conn(0x7f3ac80fdf70 msgr2=0x7f3ac8100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 -- 172.21.15.37:0/3708942709 shutdown_connections 2024-04-23T18:12:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.854+0000 7f3acd3e2700 1 -- 172.21.15.37:0/3708942709 wait complete. 2024-04-23T18:12:15.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:15 smithi037 ceph-mon[29404]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.855+0000 7f3acd3e2700 1 Processor -- start 2024-04-23T18:12:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.855+0000 7f3acd3e2700 1 -- start start 2024-04-23T18:12:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3acd3e2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3acd3e2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac810c9c0 con 0x7f3ac8108550 2024-04-23T18:12:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3ac6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:15.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3ac6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40114/0 (socket says 172.21.15.37:40114) 2024-04-23T18:12:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3ac6ffd700 1 -- 172.21.15.37:0/1223207767 learned_addr learned my addr 172.21.15.37:0/1223207767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:12:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3ac6ffd700 1 -- 172.21.15.37:0/1223207767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac81983a0 con 0x7f3ac8108550 2024-04-23T18:12:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3ac6ffd700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8197e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80056d0 tx=0x7f3ab80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ab8026070 con 0x7f3ac8108550 2024-04-23T18:12:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.856+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac8198630 con 0x7f3ac8108550 2024-04-23T18:12:15.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.857+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab800bc00 con 0x7f3ac8108550 2024-04-23T18:12:15.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.857+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac819c730 con 0x7f3ac8108550 2024-04-23T18:12:15.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.857+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ab8003920 con 0x7f3ac8108550 2024-04-23T18:12:15.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.858+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3ab8029070 con 0x7f3ac8108550 2024-04-23T18:12:15.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.858+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac8103bc0 con 0x7f3ac8108550 2024-04-23T18:12:15.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.858+0000 7f3aaffff700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3ab00396b0 0x7f3ab003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:15.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.858+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ab801e070 con 0x7f3ac8108550 2024-04-23T18:12:15.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.858+0000 7f3ac67fc700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3ab00396b0 0x7f3ab003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:15.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.858+0000 7f3ac67fc700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3ab00396b0 0x7f3ab003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3abc006fd0 tx=0x7f3abc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:15.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:15.863+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ab8013070 con 0x7f3ac8108550 2024-04-23T18:12:16.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.074+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3ac80640a0 con 0x7f3ac8108550 2024-04-23T18:12:16.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.606+0000 7f3aaffff700 1 -- 172.21.15.37:0/1223207767 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3ab800fe20 con 0x7f3ac8108550 2024-04-23T18:12:16.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.608+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3ab00396b0 msgr2=0x7f3ab003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3ab00396b0 0x7f3ab003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3abc006fd0 tx=0x7f3abc006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 msgr2=0x7f3ac8197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8197e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80056d0 tx=0x7f3ab80057b0 comp rx=0 tx=0).stop 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 shutdown_connections 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3ab00396b0 0x7f3ab003bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 --2- 172.21.15.37:0/1223207767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8108550 0x7f3ac8197e60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 >> 172.21.15.37:0/1223207767 conn(0x7f3ac80fdf70 msgr2=0x7f3ac80fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 shutdown_connections 2024-04-23T18:12:16.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:16.609+0000 7f3acd3e2700 1 -- 172.21.15.37:0/1223207767 wait complete. 2024-04-23T18:12:16.604 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-23T18:12:16.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:16 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1223207767' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:12:17.287 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-04-23T18:12:17.287 INFO:tasks.cephadm:Adding mon.b on smithi172 2024-04-23T18:12:17.287 INFO:tasks.cephadm:Adding mon.c on smithi173 2024-04-23T18:12:17.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c' 2024-04-23T18:12:17.522 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:17.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:17.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1223207767' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:12:17.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:17.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:18 smithi037 ceph-mon[29404]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:12:18.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:18 smithi037 ceph-mon[29404]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:18.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:18 smithi037 ceph-mon[29404]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:18.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:18 smithi037 ceph-mon[29404]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:12:18.996 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1309934376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 msgr2=0x7f7eb4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 --2- 172.21.15.173:0/1309934376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4101990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009910 tx=0x7f7ea8009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1309934376 shutdown_connections 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 --2- 172.21.15.173:0/1309934376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4101990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1309934376 >> 172.21.15.173:0/1309934376 conn(0x7f7eb40fce40 msgr2=0x7f7eb40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1309934376 shutdown_connections 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1309934376 wait complete. 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.002+0000 7f7eb9d5a700 1 Processor -- start 2024-04-23T18:12:18.997 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb9d5a700 1 -- start start 2024-04-23T18:12:18.999 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb9d5a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:18.999 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb9d5a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb410b830 con 0x7f7eb41015b0 2024-04-23T18:12:18.999 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:18.999 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.173:40256/0 (socket says 172.21.15.173:40256) 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb37fe700 1 -- 172.21.15.173:0/1499368516 learned_addr learned my addr 172.21.15.173:0/1499368516 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb37fe700 1 -- 172.21.15.173:0/1499368516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb4199210 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.003+0000 7f7eb37fe700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4198cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80056d0 tx=0x7f7ea80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.004+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea8026070 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.004+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb4199500 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.004+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb41999c0 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.004+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea800bba0 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.004+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea80037f0 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.005+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb4102a90 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.005+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7ea8029070 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.005+0000 7f7eb0ff9700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7e9c039650 0x7f7e9c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.005+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ea801e070 con 0x7f7eb41015b0 2024-04-23T18:12:19.000 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.005+0000 7f7eb2ffd700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7e9c039650 0x7f7e9c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:19.001 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.006+0000 7f7eb2ffd700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7e9c039650 0x7f7e9c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4006fd0 tx=0x7f7ea4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:19.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.011+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ea8013070 con 0x7f7eb41015b0 2024-04-23T18:12:19.219 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.224+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}) -- 0x7f7eb406ae60 con 0x7f7e9c039650 2024-04-23T18:12:19.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.231+0000 7f7eb0ff9700 1 -- 172.21.15.173:0/1499368516 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7eb406ae60 con 0x7f7e9c039650 2024-04-23T18:12:19.225 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2024-04-23T18:12:19.227 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7e9c039650 msgr2=0x7f7e9c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7e9c039650 0x7f7e9c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4006fd0 tx=0x7f7ea4006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 msgr2=0x7f7eb4198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4198cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80056d0 tx=0x7f7ea80057b0 comp rx=0 tx=0).stop 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 shutdown_connections 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7e9c039650 0x7f7e9c03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 --2- 172.21.15.173:0/1499368516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb41015b0 0x7f7eb4198cd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 >> 172.21.15.173:0/1499368516 conn(0x7f7eb40fce40 msgr2=0x7f7eb4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 shutdown_connections 2024-04-23T18:12:19.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:19.233+0000 7f7eb9d5a700 1 -- 172.21.15.173:0/1499368516 wait complete. 2024-04-23T18:12:19.797 DEBUG:teuthology.orchestra.run.smithi172:mon.b> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.b.service 2024-04-23T18:12:19.799 DEBUG:teuthology.orchestra.run.smithi173:mon.c> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.c.service 2024-04-23T18:12:19.802 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:12:19.802 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:12:19.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:19.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:19.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:19.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:19.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:19.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:19.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:19.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:20.521 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:20.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:20 smithi037 ceph-mon[29404]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:20.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:20 smithi037 ceph-mon[29404]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c;count:3 2024-04-23T18:12:20.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:20 smithi037 ceph-mon[29404]: Deploying daemon mon.c on smithi173 2024-04-23T18:12:21.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:21 smithi037 ceph-mon[29404]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:22 smithi037 ceph-mon[29404]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:23.150 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 systemd[1]: Starting Ceph mon.c for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1638759336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 msgr2=0x7fa4f4108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 --2- 172.21.15.173:0/1638759336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4108930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b0a0 tx=0x7fa4ec00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1638759336 shutdown_connections 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 --2- 172.21.15.173:0/1638759336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4108930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1638759336 >> 172.21.15.173:0/1638759336 conn(0x7fa4f4077ac0 msgr2=0x7fa4f4077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1638759336 shutdown_connections 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1638759336 wait complete. 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 Processor -- start 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- start start 2024-04-23T18:12:23.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.501+0000 7fa4fbe56700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f410c9b0 con 0x7fa4f4108550 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4f9bf2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4f9bf2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.173:40274/0 (socket says 172.21.15.173:40274) 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4f9bf2700 1 -- 172.21.15.173:0/1598412632 learned_addr learned my addr 172.21.15.173:0/1598412632 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4f9bf2700 1 -- 172.21.15.173:0/1598412632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4198370 con 0x7fa4f4108550 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4f9bf2700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4197e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006130 tx=0x7fa4ec0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ec021020 con 0x7fa4f4108550 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f4198600 con 0x7fa4f4108550 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ec0079b0 con 0x7fa4f4108550 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f419c700 con 0x7fa4f4108550 2024-04-23T18:12:23.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.502+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ec017040 con 0x7fa4f4108550 2024-04-23T18:12:23.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.503+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4103cf0 con 0x7fa4f4108550 2024-04-23T18:12:23.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.503+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa4ec024020 con 0x7fa4f4108550 2024-04-23T18:12:23.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.504+0000 7fa4eaffd700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa4e0039650 0x7fa4e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:23.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.504+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa4ec00f050 con 0x7fa4f4108550 2024-04-23T18:12:23.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.504+0000 7fa4f93f1700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa4e0039650 0x7fa4e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:23.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.504+0000 7fa4f93f1700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa4e0039650 0x7fa4e003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4006fd0 tx=0x7fa4e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:23.503 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.510+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4ec02b020 con 0x7fa4f4108550 2024-04-23T18:12:23.718 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 podman[32886]: 2024-04-23 18:12:23.538969966 +0000 UTC m=+0.018235331 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:12:23.718 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 podman[32886]: 2024-04-23 18:12:23.643284387 +0000 UTC m=+0.122549731 container create ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-23T18:12:23.799 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.804+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa4f40640d0 con 0x7fa4f4108550 2024-04-23T18:12:23.799 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.804+0000 7fa4eaffd700 1 -- 172.21.15.173:0/1598412632 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fa4f40640d0 con 0x7fa4f4108550 2024-04-23T18:12:23.799 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:12:23.800 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","modified":"2024-04-23T18:10:37.866561Z","created":"2024-04-23T18:10:37.866561Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T18:12:23.801 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa4e0039650 msgr2=0x7fa4e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:23.801 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa4e0039650 0x7fa4e003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4006fd0 tx=0x7fa4e4006e40 comp rx=0 tx=0).stop 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 msgr2=0x7fa4f4197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4197e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec006130 tx=0x7fa4ec0097d0 comp rx=0 tx=0).stop 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 shutdown_connections 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa4e0039650 0x7fa4e003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 --2- 172.21.15.173:0/1598412632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4f4108550 0x7fa4f4197e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 >> 172.21.15.173:0/1598412632 conn(0x7fa4f4077ac0 msgr2=0x7fa4f4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 shutdown_connections 2024-04-23T18:12:23.802 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:23.807+0000 7fa4fbe56700 1 -- 172.21.15.173:0/1598412632 wait complete. 2024-04-23T18:12:23.803 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 podman[32886]: 2024-04-23 18:12:23.804168725 +0000 UTC m=+0.283434065 container init ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 podman[32886]: 2024-04-23 18:12:23.807858434 +0000 UTC m=+0.287123774 container start ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pidfile_write: ignore empty --pid-file 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: load: jerasure load: lrc 2024-04-23T18:12:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Git sha 0 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: DB SUMMARY 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: DB Session ID: 3ZUYHTLXTQDP0NWKXPTZ 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:12:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:12:24.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:12:24.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.env: 0x555fe7eedcc0 2024-04-23T18:12:24.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:12:24.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.info_log: 0x555fea2bfbe0 2024-04-23T18:12:24.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.statistics: (nil) 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.use_fsync: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:12:24.110 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.db_log_dir: 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.wal_dir: 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.write_buffer_manager: 0x555fea2530e0 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:12:24.111 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.unordered_write: 0 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.row_cache: None 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.wal_filter: None 2024-04-23T18:12:24.112 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.wal_compression: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:12:24.113 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:12:24.114 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_open_files: -1 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Compression algorithms supported: 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kZSTD supported: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:12:24.115 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.merge_operator: 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_filter: None 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555fea2bfdc0) 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:12:24.116 INFO:journalctl@ceph.mon.c.smithi173.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: index_type: 0 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: data_block_index_type: 0 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: index_shortening: 1 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: checksum: 4 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: no_block_cache: 0 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache: 0x555fea2851f0 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:12:24.117 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache_options: 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: capacity : 536870912 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: num_shard_bits : 4 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: strict_capacity_limit : 0 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache_compressed: (nil) 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: persistent_cache: (nil) 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_size: 4096 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_size_deviation: 10 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_restart_interval: 16 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: index_block_restart_interval: 1 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: metadata_block_size: 4096 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: partition_filters: 0 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: use_delta_encoding: 1 2024-04-23T18:12:24.118 INFO:journalctl@ceph.mon.c.smithi173.stdout: filter_policy: bloomfilter 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: whole_key_filtering: 1 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: verify_compression: 0 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: format_version: 5 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: enable_index_compression: 1 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_align: 0 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: max_auto_readahead_size: 262144 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: prepopulate_block_cache: 0 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression: NoCompression 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:12:24.119 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.num_levels: 7 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:12:24.120 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:12:24.121 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:12:24.122 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:12:24.123 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.ttl: 2592000 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:12:24.124 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 59abd8df-ef07-44c4-9da7-d05e08f6f384 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713895943845807, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-23T18:12:24.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713895943846257, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713895943, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "59abd8df-ef07-44c4-9da7-d05e08f6f384", "db_session_id": "3ZUYHTLXTQDP0NWKXPTZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713895943846345, "job": 1, "event": "recovery_finished"} 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555fea39a000 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: DB pointer 0x555fea384000 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** DB Stats ** 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:12:24.126 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** Compaction Stats [default] ** 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** Compaction Stats [default] ** 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:12:24.127 INFO:journalctl@ceph.mon.c.smithi173.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Block cache BinnedLRUCache@0x555fea2851f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-23T18:12:24.128 INFO:journalctl@ceph.mon.c.smithi173.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: using public_addr v2:172.21.15.173:0/0 -> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: starting mon.c rank -1 at public addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] at bind addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(???) e0 preinit fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).mds e1 new map 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: e1 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: legacy client fscid: -1 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2024-04-23T18:12:24.129 INFO:journalctl@ceph.mon.c.smithi173.stdout: No filesystems configured 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Deploying cephadm binary to smithi173 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.130 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.131 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Added host smithi173 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1223207767' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1223207767' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:12:24.132 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:12:24.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c;count:3 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: Deploying daemon mon.c on smithi173 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:24.134 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/1598412632' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:24.135 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 ceph-mon[32906]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T18:12:24.135 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 bash[32886]: ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 2024-04-23T18:12:24.135 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:23 smithi173 systemd[1]: Started Ceph mon.c for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:12:24.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:23 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/1598412632' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:25.540 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:12:25.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:12:25.783 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:12:29.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:29.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: mon.a calling monitor election 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: mon.c calling monitor election 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: monmap epoch 2 2024-04-23T18:12:29.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: last_changed 2024-04-23T18:12:23.869102+0000 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: min_mon_release 19 (squid) 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: election_strategy: 1 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: fsmap 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: mgrmap e13: x(active, since 49s) 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: overall HEALTH_OK 2024-04-23T18:12:29.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:29.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:29.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:29.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:29.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:29.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: mon.a calling monitor election 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: mon.c calling monitor election 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:12:29.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: monmap epoch 2 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: last_changed 2024-04-23T18:12:23.869102+0000 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: min_mon_release 19 (squid) 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: election_strategy: 1 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: fsmap 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: mgrmap e13: x(active, since 49s) 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: overall HEALTH_OK 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:29.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:29.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:30.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:29 smithi173 ceph-mon[32906]: Deploying daemon mon.b on smithi172 2024-04-23T18:12:30.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 -- 172.21.15.173:0/2857793340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec100eb0 msgr2=0x7f5cd4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 --2- 172.21.15.173:0/2857793340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec100eb0 0x7f5cd4005610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005fa0 tx=0x7f5ce000ffc0 comp rx=0 tx=0).stop 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 -- 172.21.15.173:0/2857793340 shutdown_connections 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 --2- 172.21.15.173:0/2857793340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec100eb0 0x7f5cd4005610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 -- 172.21.15.173:0/2857793340 >> 172.21.15.173:0/2857793340 conn(0x7f5cec077860 msgr2=0x7f5cec077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 -- 172.21.15.173:0/2857793340 shutdown_connections 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.246+0000 7f5cf1982700 1 -- 172.21.15.173:0/2857793340 wait complete. 2024-04-23T18:12:30.241 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.247+0000 7f5cf1982700 1 Processor -- start 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.247+0000 7f5cf1982700 1 -- start start 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.247+0000 7f5cf1982700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.247+0000 7f5cf1982700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec102370 0x7f5cec1027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.247+0000 7f5cf1982700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec10a4c0 con 0x7f5cec102370 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.247+0000 7f5cf1982700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec10a340 con 0x7f5cec100eb0 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cea7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec102370 0x7f5cec1027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:55068/0 (socket says 172.21.15.173:55068) 2024-04-23T18:12:30.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 -- 172.21.15.173:0/4040419446 learned_addr learned my addr 172.21.15.173:0/4040419446 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 -- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 msgr2=0x7f5cec101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 -- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 msgr2=0x7f5cec101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5ceaffd700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cea7fc700 1 -- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 msgr2=0x7f5cec101e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cea7fc700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cea7fc700 1 -- 172.21.15.173:0/4040419446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec103d50 con 0x7f5cec102370 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cea7fc700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec102370 0x7f5cec1027f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b4d0 tx=0x7f5cdc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5cdc011630 con 0x7f5cec102370 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc011c80 con 0x7f5cec102370 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec1a4260 con 0x7f5cec102370 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5cdc010da0 con 0x7f5cec102370 2024-04-23T18:12:30.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.248+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec1a47a0 con 0x7f5cec102370 2024-04-23T18:12:30.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.249+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5cdc018020 con 0x7f5cec102370 2024-04-23T18:12:30.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.250+0000 7f5cf0980700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd80393f0 0x7f5cd803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:30.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.250+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5cdc04d130 con 0x7f5cec102370 2024-04-23T18:12:30.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.250+0000 7f5ceaffd700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd80393f0 0x7f5cd803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:30.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.250+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec06a9a0 con 0x7f5cec102370 2024-04-23T18:12:30.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.250+0000 7f5ceaffd700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd80393f0 0x7f5cd803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000fb20 tx=0x7f5ce001d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:30.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.255+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cdc01f020 con 0x7f5cec102370 2024-04-23T18:12:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:29 smithi037 ceph-mon[29404]: Deploying daemon mon.b on smithi172 2024-04-23T18:12:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:30.539 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.545+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5cec02df80 con 0x7f5cec102370 2024-04-23T18:12:30.540 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.545+0000 7f5cf0980700 1 -- 172.21.15.173:0/4040419446 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5cdc010410 con 0x7f5cec102370 2024-04-23T18:12:30.540 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:12:30.540 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","modified":"2024-04-23T18:12:23.869102Z","created":"2024-04-23T18:10:37.866561Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:12:30.542 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd80393f0 msgr2=0x7f5cd803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:30.542 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd80393f0 0x7f5cd803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000fb20 tx=0x7f5ce001d040 comp rx=0 tx=0).stop 2024-04-23T18:12:30.542 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec102370 msgr2=0x7f5cec1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:30.542 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec102370 0x7f5cec1027f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b4d0 tx=0x7f5cdc00b890 comp rx=0 tx=0).stop 2024-04-23T18:12:30.542 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 shutdown_connections 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd80393f0 0x7f5cd803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cec100eb0 0x7f5cec101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 --2- 172.21.15.173:0/4040419446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cec102370 0x7f5cec1027f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.547+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 >> 172.21.15.173:0/4040419446 conn(0x7f5cec077860 msgr2=0x7f5cec0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.548+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 shutdown_connections 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:30.548+0000 7f5cf1982700 1 -- 172.21.15.173:0/4040419446 wait complete. 2024-04-23T18:12:30.543 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-04-23T18:12:31.101 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:30 smithi173 ceph-mon[32906]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:31.102 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:30 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/4040419446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:30 smithi037 ceph-mon[29404]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:30 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/4040419446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:32.129 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:12:32.129 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:12:32.373 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:12:33.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:32 smithi172 systemd[1]: Started Ceph mon.b for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:12:38.190 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/4083151514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0d00035c0 msgr2=0x7fa0d0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:38.190 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/4083151514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0d00035c0 0x7fa0d0005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8101370 tx=0x7fa0e000fe60 comp rx=0 tx=0).stop 2024-04-23T18:12:38.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/4083151514 shutdown_connections 2024-04-23T18:12:38.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/4083151514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0d00035c0 0x7fa0d0005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/4083151514 >> 172.21.15.173:0/4083151514 conn(0x7fa0e80fcf30 msgr2=0x7fa0e80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:38.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/4083151514 shutdown_connections 2024-04-23T18:12:38.191 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.195+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/4083151514 wait complete. 2024-04-23T18:12:38.191 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:12:38.191 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.191 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: mon.a calling monitor election 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: mon.c calling monitor election 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: monmap epoch 3 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:38.192 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: last_changed 2024-04-23T18:12:32.844677+0000 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: min_mon_release 19 (squid) 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: election_strategy: 1 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: 2: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: fsmap 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: mgrmap e13: x(active, since 58s) 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: overall HEALTH_OK 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.193 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.194 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 Processor -- start 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 -- start start 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0e81972a0 0x7fa0e8197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0e81a43d0 0x7fa0e81a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e8109350 con 0x7fa0e81972a0 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81091d0 con 0x7fa0e81a43d0 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0ed2e6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81094d0 con 0x7fa0d00035c0 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.196+0000 7fa0e6ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e6ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:39790/0 (socket says 172.21.15.173:39790) 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e6ffd700 1 -- 172.21.15.173:0/2185860199 learned_addr learned my addr 172.21.15.173:0/2185860199 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:12:38.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0e81972a0 0x7fa0e8197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e6ffd700 1 -- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 msgr2=0x7fa0e8196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e6ffd700 1 -- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 msgr2=0x7fa0e8196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e6ffd700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e6ffd700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e77fe700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0e81a43d0 0x7fa0e81a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 -- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 msgr2=0x7fa0e8196d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 -- 172.21.15.173:0/2185860199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0e81a43d0 msgr2=0x7fa0e81a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0e81a43d0 0x7fa0e81a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 -- 172.21.15.173:0/2185860199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e81a6db0 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0e67fc700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0e81972a0 0x7fa0e8197720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b9b0 tx=0x7fa0dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc003c20 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0dc003dc0 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.197+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc004710 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.198+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e81a70a0 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.198+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e81a75e0 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.199+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa0dc014430 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.199+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e80640a0 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.199+0000 7fa0cffff700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0d4039ae0 0x7fa0d403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.199+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa0dc04c050 con 0x7fa0e81972a0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.199+0000 7fa0e6ffd700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0d4039ae0 0x7fa0d403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.200+0000 7fa0e6ffd700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0d4039ae0 0x7fa0d403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005d10 tx=0x7fa0e0005ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:38.199 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.204+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0dc0112a0 con 0x7fa0e81972a0 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: mon.a calling monitor election 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: mon.c calling monitor election 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: monmap epoch 3 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: last_changed 2024-04-23T18:12:32.844677+0000 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: min_mon_release 19 (squid) 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: election_strategy: 1 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: 2: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: fsmap 2024-04-23T18:12:38.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:38.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: mgrmap e13: x(active, since 58s) 2024-04-23T18:12:38.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: overall HEALTH_OK 2024-04-23T18:12:38.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:38.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.496+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa0e802df80 con 0x7fa0e81972a0 2024-04-23T18:12:38.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.496+0000 7fa0cffff700 1 -- 172.21.15.173:0/2185860199 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fa0dc00e0e0 con 0x7fa0e81972a0 2024-04-23T18:12:38.491 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T18:12:38.491 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":3,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","modified":"2024-04-23T18:12:32.844677Z","created":"2024-04-23T18:10:37.866561Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0d4039ae0 msgr2=0x7fa0d403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0d4039ae0 0x7fa0d403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005d10 tx=0x7fa0e0005ca0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0e81972a0 msgr2=0x7fa0e8197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0e81972a0 0x7fa0e8197720 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b9b0 tx=0x7fa0dc00bd70 comp rx=0 tx=0).stop 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 shutdown_connections 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0d4039ae0 0x7fa0d403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0e81a43d0 0x7fa0e81a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0d00035c0 0x7fa0e8196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 --2- 172.21.15.173:0/2185860199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0e81972a0 0x7fa0e8197720 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 >> 172.21.15.173:0/2185860199 conn(0x7fa0e80fcf30 msgr2=0x7fa0e8105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.498+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 shutdown_connections 2024-04-23T18:12:38.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:38.499+0000 7fa0ed2e6700 1 -- 172.21.15.173:0/2185860199 wait complete. 2024-04-23T18:12:38.494 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 3 2024-04-23T18:12:39.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:38 smithi173 ceph-mon[32906]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:39.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:38 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:39.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/2185860199' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:39.030 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:38 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:39.058 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T18:12:39.058 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T18:12:39.314 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:38 smithi037 ceph-mon[29404]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:39.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:38 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:39.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/2185860199' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:39.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:38 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: mon.a calling monitor election 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: mon.c calling monitor election 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: monmap epoch 3 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: last_changed 2024-04-23T18:12:32.844677+0000 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: min_mon_release 19 (squid) 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: election_strategy: 1 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:40.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: 2: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: fsmap 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: mgrmap e13: x(active, since 58s) 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: overall HEALTH_OK 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/2185860199' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:12:40.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.956+0000 7f839a81a700 1 -- 172.21.15.37:0/3448721443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83941015b0 msgr2=0x7f8394101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.956+0000 7f839a81a700 1 --2- 172.21.15.37:0/3448721443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83941015b0 0x7f8394101990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8384009910 tx=0x7f8384009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.956+0000 7f839a81a700 1 -- 172.21.15.37:0/3448721443 shutdown_connections 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.956+0000 7f839a81a700 1 --2- 172.21.15.37:0/3448721443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83941015b0 0x7f8394101990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.956+0000 7f839a81a700 1 -- 172.21.15.37:0/3448721443 >> 172.21.15.37:0/3448721443 conn(0x7f83940fce60 msgr2=0x7f83940ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.956+0000 7f839a81a700 1 -- 172.21.15.37:0/3448721443 shutdown_connections 2024-04-23T18:12:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.957+0000 7f839a81a700 1 -- 172.21.15.37:0/3448721443 wait complete. 2024-04-23T18:12:40.951 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: mon.b calling monitor election 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: mon.b calling monitor election 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: mon.c calling monitor election 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: mon.a calling monitor election 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: monmap epoch 3 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: last_changed 2024-04-23T18:12:32.844677+0000 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: min_mon_release 19 (squid) 2024-04-23T18:12:40.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: election_strategy: 1 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: 2: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: fsmap 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: mgrmap e13: x(active, since 60s) 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: overall HEALTH_OK 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:40.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:40 smithi037 ceph-mon[29404]: Reconfiguring daemon mon.a on smithi037 2024-04-23T18:12:40.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.957+0000 7f839a81a700 1 Processor -- start 2024-04-23T18:12:40.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 -- start start 2024-04-23T18:12:40.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83941015b0 0x7f839419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:40.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 0x7f83941a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:40.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83941a3550 0x7f83941a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:40.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f839410b830 con 0x7f839419ec80 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f839410b6b0 con 0x7f83941a3550 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f839a81a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f839410b9b0 con 0x7f83941015b0 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f83937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 0x7f83941a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f83937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 0x7f83941a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58962/0 (socket says 172.21.15.37:58962) 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f83937fe700 1 -- 172.21.15.37:0/116162420 learned_addr learned my addr 172.21.15.37:0/116162420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f8398db7700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83941a3550 0x7f83941a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.958+0000 7f8393fff700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83941015b0 0x7f839419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83937fe700 1 -- 172.21.15.37:0/116162420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83941015b0 msgr2=0x7f839419e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83937fe700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83941015b0 0x7f839419e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83937fe700 1 -- 172.21.15.37:0/116162420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83941a3550 msgr2=0x7f83941a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83937fe700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83941a3550 0x7f83941a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83937fe700 1 -- 172.21.15.37:0/116162420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83941ab2c0 con 0x7f839419ec80 2024-04-23T18:12:40.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83937fe700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 0x7f83941a3010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f838800b780 tx=0x7f838800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388004230 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83880043d0 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388011b20 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83941ab4f0 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.959+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83941aba30 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.960+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8388011cc0 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.960+0000 7f83917fa700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f837c039ae0 0x7f837c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.960+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83940640a0 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.961+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f838804c340 con 0x7f839419ec80 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.961+0000 7f8393fff700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f837c039ae0 0x7f837c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:40.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.961+0000 7f8393fff700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f837c039ae0 0x7f837c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8384009750 tx=0x7f838401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:40.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:40.966+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8388020500 con 0x7f839419ec80 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: mon.b calling monitor election 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: mon.b calling monitor election 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: mon.c calling monitor election 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: mon.a calling monitor election 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: monmap epoch 3 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:41.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: last_changed 2024-04-23T18:12:32.844677+0000 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: min_mon_release 19 (squid) 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: election_strategy: 1 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: 2: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: fsmap 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: mgrmap e13: x(active, since 60s) 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: overall HEALTH_OK 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:41.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:41.091 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:40 smithi172 ceph-mon[32456]: Reconfiguring daemon mon.a on smithi037 2024-04-23T18:12:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: mon.b calling monitor election 2024-04-23T18:12:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:12:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: mon.b calling monitor election 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: mon.c calling monitor election 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: mon.a calling monitor election 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: monmap epoch 3 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: last_changed 2024-04-23T18:12:32.844677+0000 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: created 2024-04-23T18:10:37.866561+0000 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: min_mon_release 19 (squid) 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: election_strategy: 1 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-23T18:12:41.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: 2: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: fsmap 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: mgrmap e13: x(active, since 60s) 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: overall HEALTH_OK 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.108 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:41.109 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:40 smithi173 ceph-mon[32906]: Reconfiguring daemon mon.a on smithi037 2024-04-23T18:12:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.183+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f83941abc90 con 0x7f839419ec80 2024-04-23T18:12:41.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.184+0000 7f83917fa700 1 -- 172.21.15.37:0/116162420 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f83880202a0 con 0x7f839419ec80 2024-04-23T18:12:41.178 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:41.178 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-23T18:12:41.178 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:12:41.178 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-04-23T18:12:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.186+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f837c039ae0 msgr2=0x7f837c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.186+0000 7f839a81a700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f837c039ae0 0x7f837c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8384009750 tx=0x7f838401a040 comp rx=0 tx=0).stop 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.186+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 msgr2=0x7f83941a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.186+0000 7f839a81a700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 0x7f83941a3010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f838800b780 tx=0x7f838800ba90 comp rx=0 tx=0).stop 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 shutdown_connections 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f837c039ae0 0x7f837c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83941a3550 0x7f83941a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83941015b0 0x7f839419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 --2- 172.21.15.37:0/116162420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839419ec80 0x7f83941a3010 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 >> 172.21.15.37:0/116162420 conn(0x7f83940fce60 msgr2=0x7f83941091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 shutdown_connections 2024-04-23T18:12:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:41.187+0000 7f839a81a700 1 -- 172.21.15.37:0/116162420 wait complete. 2024-04-23T18:12:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:41 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/116162420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:41 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/116162420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:12:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:41 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/116162420' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:42.496 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T18:12:42.496 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:12:42.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:12:42.523 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:12:42.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:42.587 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:12:42.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:12:42.613 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:12:42.613 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:42.676 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:12:42.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:12:42.710 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:12:42.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:12:42.780 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-04-23T18:12:42.780 INFO:tasks.cephadm:Adding mgr.y on smithi172 2024-04-23T18:12:42.780 INFO:tasks.cephadm:Adding mgr.z on smithi173 2024-04-23T18:12:42.780 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi172=y;smithi173=z' 2024-04-23T18:12:43.055 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:12:43.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:43.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:43 smithi172 ceph-mon[32456]: Reconfiguring daemon mon.b on smithi172 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:43.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:43 smithi173 ceph-mon[32906]: Reconfiguring daemon mon.b on smithi172 2024-04-23T18:12:43.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:43.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:43.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:43.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:12:43.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:43.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:43.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:43.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:43 smithi037 ceph-mon[29404]: Reconfiguring daemon mon.b on smithi172 2024-04-23T18:12:44.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:44.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:44.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:12:44.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:44.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:44.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:44.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:44 smithi172 ceph-mon[32456]: Reconfiguring daemon mon.c on smithi173 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:44.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:44 smithi173 ceph-mon[32906]: Reconfiguring daemon mon.c on smithi173 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:44 smithi037 ceph-mon[29404]: Reconfiguring daemon mon.c on smithi173 2024-04-23T18:12:45.193 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 -- 172.21.15.173:0/3322013001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91141014e0 msgr2=0x7f9114101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:45.193 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 --2- 172.21.15.173:0/3322013001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91141014e0 0x7f9114101900 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9104009910 tx=0x7f9104009c20 comp rx=0 tx=0).stop 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 -- 172.21.15.173:0/3322013001 shutdown_connections 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 --2- 172.21.15.173:0/3322013001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91141014e0 0x7f9114101900 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 -- 172.21.15.173:0/3322013001 >> 172.21.15.173:0/3322013001 conn(0x7f91140fca80 msgr2=0x7f91140feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 -- 172.21.15.173:0/3322013001 shutdown_connections 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.197+0000 7f9118985700 1 -- 172.21.15.173:0/3322013001 wait complete. 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 Processor -- start 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 -- start start 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91141014e0 0x7f911406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 0x7f911419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911419d450 0x7f911419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9114107420 con 0x7f911419d450 2024-04-23T18:12:45.194 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f91141072a0 con 0x7f91141014e0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9118985700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f91141075a0 con 0x7f911419aad0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 0x7f911419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 0x7f911419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:39792/0 (socket says 172.21.15.173:39792) 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 -- 172.21.15.173:0/3104152390 learned_addr learned my addr 172.21.15.173:0/3104152390 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f911259c700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91141014e0 0x7f911406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 -- 172.21.15.173:0/3104152390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91141014e0 msgr2=0x7f911406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91141014e0 0x7f911406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 -- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911419d450 msgr2=0x7f911419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911419d450 0x7f911419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 -- 172.21.15.173:0/3104152390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a01b0 con 0x7f911419aad0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.198+0000 7f9111d9b700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 0x7f911419cec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f910800b780 tx=0x7f910800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.199+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108004230 con 0x7f911419aad0 2024-04-23T18:12:45.195 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.199+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a04a0 con 0x7f911419aad0 2024-04-23T18:12:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.199+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91080043d0 con 0x7f911419aad0 2024-04-23T18:12:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.199+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108011bd0 con 0x7f911419aad0 2024-04-23T18:12:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.199+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a0a60 con 0x7f911419aad0 2024-04-23T18:12:45.196 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.199+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f910801a400 con 0x7f911419aad0 2024-04-23T18:12:45.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.200+0000 7f91037fe700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f90fc039ae0 0x7f90fc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:45.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.200+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f910804c510 con 0x7f911419aad0 2024-04-23T18:12:45.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.200+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91140640a0 con 0x7f911419aad0 2024-04-23T18:12:45.197 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.200+0000 7f911259c700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f90fc039ae0 0x7f90fc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:45.198 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.201+0000 7f911259c700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f90fc039ae0 0x7f90fc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9104005950 tx=0x7f910401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:45.199 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.205+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9108018350 con 0x7f911419aad0 2024-04-23T18:12:45.415 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.421+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi172=y;smithi173=z", "target": ["mon-mgr", ""]}) -- 0x7f9114062e80 con 0x7f90fc039ae0 2024-04-23T18:12:45.422 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.428+0000 7f91037fe700 1 -- 172.21.15.173:0/3104152390 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9114062e80 con 0x7f90fc039ae0 2024-04-23T18:12:45.422 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mgr update... 2024-04-23T18:12:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f90fc039ae0 msgr2=0x7f90fc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f90fc039ae0 0x7f90fc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9104005950 tx=0x7f910401a040 comp rx=0 tx=0).stop 2024-04-23T18:12:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 msgr2=0x7f911419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 0x7f911419cec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f910800b780 tx=0x7f910800bb40 comp rx=0 tx=0).stop 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 shutdown_connections 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f90fc039ae0 0x7f90fc03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911419d450 0x7f911419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91141014e0 0x7f911406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 --2- 172.21.15.173:0/3104152390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f911419aad0 0x7f911419cec0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 >> 172.21.15.173:0/3104152390 conn(0x7f91140fca80 msgr2=0x7f91140fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 shutdown_connections 2024-04-23T18:12:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:12:45.430+0000 7f9118985700 1 -- 172.21.15.173:0/3104152390 wait complete. 2024-04-23T18:12:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:45 smithi173 ceph-mon[32906]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:45 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:45.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:45 smithi037 ceph-mon[29404]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:45.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:45 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:45 smithi172 ceph-mon[32456]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:46.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:45 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.240 DEBUG:teuthology.orchestra.run.smithi172:mgr.y> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.y.service 2024-04-23T18:12:46.243 DEBUG:teuthology.orchestra.run.smithi173:mgr.z> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.z.service 2024-04-23T18:12:46.246 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T18:12:46.246 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:12:46.246 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:12:46.262 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:12:46.262 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-23 18:11:50.059907503 +0000 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-23 18:06:37.985695654 +0000 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-23 18:06:37.985695654 +0000 2024-04-23T18:12:46.318 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-23 18:06:37.985695654 +0000 2024-04-23T18:12:46.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:12:46.384 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-23T18:12:46.384 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-23T18:12:46.385 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000414704 s, 1.2 MB/s 2024-04-23T18:12:46.386 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:12:46.442 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-23T18:12:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:12:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:46.497 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T18:12:46.497 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:46.498 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:46.498 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-23 18:11:50.096907006 +0000 2024-04-23T18:12:46.498 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-23 18:06:38.341690815 +0000 2024-04-23T18:12:46.498 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-23 18:06:38.341690815 +0000 2024-04-23T18:12:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-23 18:06:38.341690815 +0000 2024-04-23T18:12:46.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:12:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-23T18:12:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-23T18:12:46.561 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000192384 s, 2.7 MB/s 2024-04-23T18:12:46.562 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:12:46.618 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-23T18:12:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-23 18:11:50.130906551 +0000 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-23 18:06:38.692686044 +0000 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-23 18:06:38.692686044 +0000 2024-04-23T18:12:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-23 18:06:38.692686044 +0000 2024-04-23T18:12:46.673 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi172=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: Saving service mgr spec with placement smithi037=x;smithi172=y;smithi173=z;count:3 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:46.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:46.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:46.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:12:46.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:46.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:46.736 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-23T18:12:46.736 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-23T18:12:46.736 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000190834 s, 2.7 MB/s 2024-04-23T18:12:46.737 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:12:46.793 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-23 18:11:50.165906081 +0000 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-23 18:06:39.038681341 +0000 2024-04-23T18:12:46.849 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-23 18:06:39.038681341 +0000 2024-04-23T18:12:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-23 18:06:39.038681341 +0000 2024-04-23T18:12:46.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi172=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: Saving service mgr spec with placement smithi037=x;smithi172=y;smithi173=z;count:3 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:12:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:46.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-23T18:12:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-23T18:12:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000209217 s, 2.4 MB/s 2024-04-23T18:12:46.914 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:12:46.971 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-23T18:12:46.971 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:12:46.990 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:12:46.991 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-04-23T18:12:47.049 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:12:47.050 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:47.050 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-23T18:12:47.050 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:47.050 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:47.050 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-23 18:12:32.293745083 +0000 2024-04-23T18:12:47.051 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-23 18:07:00.330252645 +0000 2024-04-23T18:12:47.051 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-23 18:07:00.330252645 +0000 2024-04-23T18:12:47.051 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-23 18:07:00.330252645 +0000 2024-04-23T18:12:47.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:12:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi172=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: Saving service mgr spec with placement smithi037=x;smithi172=y;smithi173=z;count:3 2024-04-23T18:12:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:47.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:47.114 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-23T18:12:47.114 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-23T18:12:47.114 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000282798 s, 1.8 MB/s 2024-04-23T18:12:47.116 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:12:47.173 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-04-23T18:12:47.230 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:12:47.230 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-23 18:12:32.293745083 +0000 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-23 18:07:00.667244051 +0000 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-23 18:07:00.667244051 +0000 2024-04-23T18:12:47.231 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-23 18:07:00.667244051 +0000 2024-04-23T18:12:47.231 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:12:47.295 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-23T18:12:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-23T18:12:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000225779 s, 2.3 MB/s 2024-04-23T18:12:47.297 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:12:47.353 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-23 18:12:32.294745057 +0000 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-23 18:07:01.000235561 +0000 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-23 18:07:01.000235561 +0000 2024-04-23T18:12:47.411 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-23 18:07:01.000235561 +0000 2024-04-23T18:12:47.411 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:12:47.474 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-23T18:12:47.474 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-23T18:12:47.474 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000207012 s, 2.5 MB/s 2024-04-23T18:12:47.475 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:12:47.532 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-23 18:12:32.294745057 +0000 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-23 18:07:01.315227529 +0000 2024-04-23T18:12:47.589 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-23 18:07:01.315227529 +0000 2024-04-23T18:12:47.590 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-23 18:07:01.315227529 +0000 2024-04-23T18:12:47.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:12:47.653 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-23T18:12:47.654 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-23T18:12:47.654 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000200165 s, 2.6 MB/s 2024-04-23T18:12:47.655 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:12:47.711 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T18:12:47.711 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:12:47.728 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:12:47.729 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 18:12:23.065059260 +0000 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 18:06:51.131918619 +0000 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 18:06:51.131918619 +0000 2024-04-23T18:12:47.786 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 18:06:51.131918619 +0000 2024-04-23T18:12:47.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:12:47.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:47 smithi172 ceph-mon[32456]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:47.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:47 smithi172 ceph-mon[32456]: Deploying daemon mgr.z on smithi173 2024-04-23T18:12:47.849 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:47 smithi173 ceph-mon[32906]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:47.849 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:47 smithi173 ceph-mon[32906]: Deploying daemon mgr.z on smithi173 2024-04-23T18:12:47.852 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T18:12:47.852 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T18:12:47.852 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000255679 s, 2.0 MB/s 2024-04-23T18:12:47.853 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:12:47.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:47 smithi037 ceph-mon[29404]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:47.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:47 smithi037 ceph-mon[29404]: Deploying daemon mgr.z on smithi173 2024-04-23T18:12:47.911 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-04-23T18:12:47.967 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:12:47.967 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:47.967 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-23T18:12:47.967 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:47.967 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:47.968 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 18:12:23.065059260 +0000 2024-04-23T18:12:47.968 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 18:06:51.455911456 +0000 2024-04-23T18:12:47.968 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 18:06:51.455911456 +0000 2024-04-23T18:12:47.968 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 18:06:51.455911456 +0000 2024-04-23T18:12:47.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:12:48.129 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T18:12:48.129 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T18:12:48.130 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000208025 s, 2.5 MB/s 2024-04-23T18:12:48.131 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:12:48.146 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 18:12:48.091517255 +0000 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 18:06:51.811903587 +0000 2024-04-23T18:12:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 18:06:51.811903587 +0000 2024-04-23T18:12:48.208 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 18:06:51.811903587 +0000 2024-04-23T18:12:48.208 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:12:48.312 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T18:12:48.312 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T18:12:48.312 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000187137 s, 2.7 MB/s 2024-04-23T18:12:48.313 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:12:48.329 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 18:12:48.276513249 +0000 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 18:06:52.163895806 +0000 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 18:06:52.163895806 +0000 2024-04-23T18:12:48.385 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 18:06:52.163895806 +0000 2024-04-23T18:12:48.385 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:12:48.446 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 systemd[1]: Starting Ceph mgr.z for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:12:48.449 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T18:12:48.449 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T18:12:48.450 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000220275 s, 2.3 MB/s 2024-04-23T18:12:48.451 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:12:48.508 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-04-23T18:12:48.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:12:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:48.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 podman[34733]: 2024-04-23 18:12:48.559120036 +0000 UTC m=+0.107754973 container create 3348114d907dc1e104a0fe47c2827f3faf905cf48b42a680e3a24cdfecac955d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:12:48.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 podman[34733]: 2024-04-23 18:12:48.469639391 +0000 UTC m=+0.018274287 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:12:49.201 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 podman[34733]: 2024-04-23 18:12:48.869569286 +0000 UTC m=+0.418204189 container init 3348114d907dc1e104a0fe47c2827f3faf905cf48b42a680e3a24cdfecac955d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T18:12:49.201 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 podman[34733]: 2024-04-23 18:12:48.872180195 +0000 UTC m=+0.420815089 container start 3348114d907dc1e104a0fe47c2827f3faf905cf48b42a680e3a24cdfecac955d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:12:49.201 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 ceph-mgr[34773]: -- 172.21.15.173:0/419584036 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a1a7665a0 con 0x560a1a73ac00 2024-04-23T18:12:49.202 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 bash[34733]: 3348114d907dc1e104a0fe47c2827f3faf905cf48b42a680e3a24cdfecac955d 2024-04-23T18:12:49.202 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:48 smithi173 systemd[1]: Started Ceph mgr.z for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:12:49.202 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:49.022+0000 7f070b9a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:12:49.202 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:49.207+0000 7f070b9a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:12:49.600 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:49.309+0000 7f070b9a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:12:49.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:49.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:49.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:12:49.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:49.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:49.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:49.621+0000 7f070b9a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:12:49.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:49.704+0000 7f070b9a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:12:49.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:49.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:49.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:12:49.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:49.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:49.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:49.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:49.923 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:49.924 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:12:49.924 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:49.924 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:12:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:12:50.635 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-23T18:12:50.635 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-23T18:12:50.635 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111996 s, 936 MB/s 2024-04-23T18:12:50.635 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-23T18:12:50.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:50 smithi173 ceph-mon[32906]: Deploying daemon mgr.y on smithi172 2024-04-23T18:12:50.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:50 smithi037 ceph-mon[29404]: Deploying daemon mgr.y on smithi172 2024-04-23T18:12:50.983 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:50 smithi172 ceph-mon[32456]: Deploying daemon mgr.y on smithi172 2024-04-23T18:12:51.193 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-04-23T18:12:51.258 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:50 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:50.915+0000 7f070b9a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:12:51.258 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:51.088+0000 7f070b9a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:12:51.258 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:51.263+0000 7f070b9a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:12:51.434 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:12:51.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:51.793+0000 7fc6033f1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:12:51.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-mon[32456]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:51.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.850 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-mon[32906]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:51.850 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.851 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.851 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.851 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.851 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:51.554+0000 7f070b9a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:12:51.851 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:51.647+0000 7f070b9a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:12:51.851 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:51.856+0000 7f070b9a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:12:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:51 smithi037 ceph-mon[29404]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:52.106 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:52.057+0000 7f070b9a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:12:52.228 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:51.907+0000 7fc6033f1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:12:52.228 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:52.234+0000 7fc6033f1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:12:52.559 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:52.334+0000 7fc6033f1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:12:52.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:52.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:52.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:52.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:52.692+0000 7f070b9a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 -- 172.21.15.37:0/2872998081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 msgr2=0x7f81c4105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 --2- 172.21.15.37:0/2872998081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4105980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00b0a0 tx=0x7f81bc00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 -- 172.21.15.37:0/2872998081 shutdown_connections 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 --2- 172.21.15.37:0/2872998081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4105980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 --2- 172.21.15.37:0/2872998081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81c41087f0 0x7f81c410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 --2- 172.21.15.37:0/2872998081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81c4105ec0 0x7f81c41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:52.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 -- 172.21.15.37:0/2872998081 >> 172.21.15.37:0/2872998081 conn(0x7f81c40fcf40 msgr2=0x7f81c40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:12:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 -- 172.21.15.37:0/2872998081 shutdown_connections 2024-04-23T18:12:52.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.888+0000 7f81c9a50700 1 -- 172.21.15.37:0/2872998081 wait complete. 2024-04-23T18:12:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 Processor -- start 2024-04-23T18:12:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 -- start start 2024-04-23T18:12:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:52.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81c4105ec0 0x7f81c4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81c41087f0 0x7f81c41a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81c410e530 con 0x7f81c4103590 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81c410e3b0 con 0x7f81c4105ec0 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c9a50700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81c410e6b0 con 0x7f81c41087f0 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47636/0 (socket says 172.21.15.37:47636) 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 -- 172.21.15.37:0/699300866 learned_addr learned my addr 172.21.15.37:0/699300866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c27fc700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81c4105ec0 0x7f81c4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c37fe700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81c41087f0 0x7f81c41a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 -- 172.21.15.37:0/699300866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81c41087f0 msgr2=0x7f81c41a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81c41087f0 0x7f81c41a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 -- 172.21.15.37:0/699300866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81c4105ec0 msgr2=0x7f81c4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81c4105ec0 0x7f81c4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.889+0000 7f81c2ffd700 1 -- 172.21.15.37:0/699300866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41a08a0 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.890+0000 7f81c2ffd700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4198c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00b780 tx=0x7f81bc004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.890+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc021070 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.890+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc007b80 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.890+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c41a0b30 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.890+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0052a0 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.890+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a0fe0 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.891+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f81bc024070 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.891+0000 7f81c8a4e700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f81b0039890 0x7f81b003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.891+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c4069730 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.891+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f81bc00f020 con 0x7f81c4103590 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.891+0000 7f81c27fc700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f81b0039890 0x7f81b003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:12:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.892+0000 7f81c27fc700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f81b0039890 0x7f81b003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009c60 tx=0x7f81ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:12:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:52.896+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81bc02b020 con 0x7f81c4103590 2024-04-23T18:12:53.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:53.115+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f81c4062e80 con 0x7f81b0039890 2024-04-23T18:12:53.420 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:53.129+0000 7f070b9a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:12:53.421 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:53.225+0000 7f070b9a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:12:53.421 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:53.426+0000 7f070b9a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:12:53.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:53.589+0000 7fc6033f1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:12:53.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:53.759+0000 7fc6033f1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:12:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:12:53.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:53.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:53.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:12:53.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:54.209 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:53.930+0000 7fc6033f1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:12:54.210 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:54.215+0000 7fc6033f1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:12:54.282 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:53.955+0000 7f070b9a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:12:54.283 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:54.047+0000 7f070b9a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:12:54.503 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:54.306+0000 7fc6033f1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:12:54.503 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:54.509+0000 7fc6033f1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:12:54.606 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:54.288+0000 7f070b9a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:12:54.606 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:54.418+0000 7f070b9a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:12:54.606 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:54.512+0000 7f070b9a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:12:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-mon[32456]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-mon[32456]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:12:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-mon[32456]: Reconfiguring daemon mgr.x on smithi037 2024-04-23T18:12:54.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:54.701+0000 7fc6033f1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:12:54.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:54 smithi037 ceph-mon[29404]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:54 smithi037 ceph-mon[29404]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:12:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:54 smithi037 ceph-mon[29404]: Reconfiguring daemon mgr.x on smithi037 2024-04-23T18:12:54.932 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:54.630+0000 7f070b9a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:12:54.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-mon[32906]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:12:54.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-mon[32906]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:12:54.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-mon[32906]: Reconfiguring daemon mgr.x on smithi037 2024-04-23T18:12:55.355 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:54.938+0000 7f070b9a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:12:55.588 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:55.297+0000 7fc6033f1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:12:55.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:55 smithi037 ceph-mon[29404]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:55.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:55.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:55 smithi173 ceph-mon[32906]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:55.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:55.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:55.533+0000 7f070b9a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:12:55.856 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:12:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:12:55.750+0000 7f070b9a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:12:56.008 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:55 smithi172 ceph-mon[32456]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:56.008 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:56.008 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:55.713+0000 7fc6033f1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:12:56.008 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:55.811+0000 7fc6033f1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:12:56.008 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:56.013+0000 7fc6033f1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:12:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:56.632+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f81bc012cd0 con 0x7f81c4103590 2024-04-23T18:12:56.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:56.550+0000 7fc6033f1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:12:56.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:56.644+0000 7fc6033f1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:12:56.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: Standby manager daemon z started 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:56.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:56.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: Standby manager daemon z started 2024-04-23T18:12:56.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:56.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: Standby manager daemon z started 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/281745375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:57.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:57.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:12:57.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:12:57.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:12:57.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:12:57.218 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:56.884+0000 7fc6033f1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:12:57.218 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:57.016+0000 7fc6033f1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:12:57.218 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:57.109+0000 7fc6033f1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:12:57.218 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:57.224+0000 7fc6033f1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:12:57.838 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:57.527+0000 7fc6033f1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:12:57.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-23T18:12:57.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:12:57.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2893636802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d65e5f77-8366-4dbf-92bf-a34c53c514c5"}]: dispatch 2024-04-23T18:12:57.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2893636802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d65e5f77-8366-4dbf-92bf-a34c53c514c5"}]': finished 2024-04-23T18:12:57.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:12:57.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:12:57.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3116328554' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:12:58.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-23T18:12:58.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:12:58.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2893636802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d65e5f77-8366-4dbf-92bf-a34c53c514c5"}]: dispatch 2024-04-23T18:12:58.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2893636802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d65e5f77-8366-4dbf-92bf-a34c53c514c5"}]': finished 2024-04-23T18:12:58.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:12:58.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:12:58.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3116328554' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:12:58.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-23T18:12:58.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:12:58.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2893636802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d65e5f77-8366-4dbf-92bf-a34c53c514c5"}]: dispatch 2024-04-23T18:12:58.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2893636802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d65e5f77-8366-4dbf-92bf-a34c53c514c5"}]': finished 2024-04-23T18:12:58.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:12:58.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:12:58.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3116328554' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:12:58.588 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:58.127+0000 7fc6033f1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:12:58.588 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:12:58.341+0000 7fc6033f1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:12:58.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:12:58.649+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f81bc01d450 con 0x7f81c4103590 2024-04-23T18:12:59.049 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:58 smithi037 ceph-mon[29404]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:59.049 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:58 smithi037 ceph-mon[29404]: Standby manager daemon y started 2024-04-23T18:12:59.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:58 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:12:59.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:58 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:12:59.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:58 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:12:59.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:58 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:12:59.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-mon[32456]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:59.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-mon[32456]: Standby manager daemon y started 2024-04-23T18:12:59.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:12:59.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:12:59.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:12:59.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:58 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:12:59.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:58 smithi173 ceph-mon[32906]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:12:59.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:58 smithi173 ceph-mon[32906]: Standby manager daemon y started 2024-04-23T18:12:59.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:58 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:12:59.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:58 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:12:59.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:58 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:12:59.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:58 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/483414598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:13:00.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:59 smithi037 ceph-mon[29404]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-23T18:13:00.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:12:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:13:00.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:59 smithi172 ceph-mon[32456]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-23T18:13:00.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:12:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:13:00.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:59 smithi173 ceph-mon[32906]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-23T18:13:00.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:12:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:13:00.905 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:00 smithi173 ceph-mon[32906]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:01.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:00 smithi172 ceph-mon[32456]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:01.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:00 smithi037 ceph-mon[29404]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:03.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:02 smithi172 ceph-mon[32456]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:03.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:02 smithi173 ceph-mon[32906]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:03.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:02 smithi037 ceph-mon[29404]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:04 smithi173 ceph-mon[32906]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:13:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:04 smithi173 ceph-mon[32906]: Deploying daemon osd.0 on smithi037 2024-04-23T18:13:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:04 smithi037 ceph-mon[29404]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:13:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:04 smithi037 ceph-mon[29404]: Deploying daemon osd.0 on smithi037 2024-04-23T18:13:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:04 smithi172 ceph-mon[32456]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:13:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:04 smithi172 ceph-mon[32456]: Deploying daemon osd.0 on smithi037 2024-04-23T18:13:07.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:06 smithi173 ceph-mon[32906]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:07.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:06 smithi037 ceph-mon[29404]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:07.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:06 smithi172 ceph-mon[32456]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:08.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:08 smithi037 ceph-mon[29404]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:09.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:08 smithi172 ceph-mon[32456]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:09.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:08 smithi173 ceph-mon[32906]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:10.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:10.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:10.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:10.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:10.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:10.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:11.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:11 smithi173 ceph-mon[32906]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:11.868 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:11 smithi037 ceph-mon[29404]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:12.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:11 smithi172 ceph-mon[32456]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:12.921 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-04-23T18:13:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.925+0000 7f81c8a4e700 1 -- 172.21.15.37:0/699300866 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81c4062e80 con 0x7f81b0039890 2024-04-23T18:13:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f81b0039890 msgr2=0x7f81b003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f81b0039890 0x7f81b003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009c60 tx=0x7f81ac009260 comp rx=0 tx=0).stop 2024-04-23T18:13:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 msgr2=0x7f81c4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4198c50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00b780 tx=0x7f81bc004d70 comp rx=0 tx=0).stop 2024-04-23T18:13:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 shutdown_connections 2024-04-23T18:13:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f81b0039890 0x7f81b003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81c4103590 0x7f81c4198c50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81c41087f0 0x7f81c41a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 --2- 172.21.15.37:0/699300866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81c4105ec0 0x7f81c4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.928+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 >> 172.21.15.37:0/699300866 conn(0x7f81c40fcf40 msgr2=0x7f81c4105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:13:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.929+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 shutdown_connections 2024-04-23T18:13:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:13:12.929+0000 7f81c9a50700 1 -- 172.21.15.37:0/699300866 wait complete. 2024-04-23T18:13:13.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:13.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:13 smithi037 ceph-mon[29404]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:13:13.560 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T18:13:13.563 INFO:tasks.cephadm:Deploying osd.1 on smithi172 with /dev/vg_nvme/lv_4... 2024-04-23T18:13:13.564 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:13:13.821 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:13.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:13 smithi172 ceph-mon[32456]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:13 smithi173 ceph-mon[32906]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:13:14.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-mon[29404]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:14.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-mon[29404]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:13:14.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-mon[29404]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:13:14.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-mon[29404]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:13:14.990 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:14.990 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0[41855]: 2024-04-23T18:13:14.920+0000 7fe8d870e700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T18:13:14.990 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:13:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0[41855]: 2024-04-23T18:13:14.925+0000 7fe8d4501700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:13:15.211 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:14 smithi172 ceph-mon[32456]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:15.211 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:14 smithi172 ceph-mon[32456]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:13:15.211 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:14 smithi172 ceph-mon[32456]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:13:15.211 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:14 smithi172 ceph-mon[32456]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:13:15.211 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:14 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:15.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:14 smithi173 ceph-mon[32906]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:15.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:14 smithi173 ceph-mon[32906]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:13:15.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:14 smithi173 ceph-mon[32906]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:13:15.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:14 smithi173 ceph-mon[32906]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:13:15.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:14 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:15.803 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:13:15.803 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:13:15.803 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-23T18:13:15.804 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-23T18:13:15.804 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134493 s, 780 MB/s 2024-04-23T18:13:15.804 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-23T18:13:16.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:15 smithi037 ceph-mon[29404]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:13:16.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:15 smithi037 ceph-mon[29404]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:13:16.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:15 smithi172 ceph-mon[32456]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:13:16.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:15 smithi172 ceph-mon[32456]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:13:16.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:15 smithi173 ceph-mon[32906]: from='osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:13:16.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:15 smithi173 ceph-mon[32906]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:13:16.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.444 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-04-23T18:13:16.694 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] boot 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:16.987 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi037 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:16.988 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:13:17.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:13:17.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:17.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:17.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] boot 2024-04-23T18:13:17.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi037 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:17.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:13:17.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: osd.0 [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] boot 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi037 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:17.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:17.973 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:17 smithi172 ceph-mon[32456]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 -- 172.21.15.172:0/3589978205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c108790 msgr2=0x7f0a2c10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 --2- 172.21.15.172:0/3589978205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c108790 0x7f0a2c10ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009990 tx=0x7f0a24009ca0 comp rx=0 tx=0).stop 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 -- 172.21.15.172:0/3589978205 shutdown_connections 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 --2- 172.21.15.172:0/3589978205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c108790 0x7f0a2c10ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 --2- 172.21.15.172:0/3589978205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a2c103530 0x7f0a2c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 --2- 172.21.15.172:0/3589978205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c105e60 0x7f0a2c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 -- 172.21.15.172:0/3589978205 >> 172.21.15.172:0/3589978205 conn(0x7f0a2c0fcf40 msgr2=0x7f0a2c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 -- 172.21.15.172:0/3589978205 shutdown_connections 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 -- 172.21.15.172:0/3589978205 wait complete. 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.219+0000 7f0a314a0700 1 Processor -- start 2024-04-23T18:13:18.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 -- start start 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a2c103530 0x7f0a2c102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 0x7f0a2c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c108790 0x7f0a2c101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10e5d0 con 0x7f0a2c108790 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10e450 con 0x7f0a2c105e60 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a314a0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10e750 con 0x7f0a2c103530 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 0x7f0a2c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 0x7f0a2c1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58754/0 (socket says 172.21.15.172:58754) 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 -- 172.21.15.172:0/2694106831 learned_addr learned my addr 172.21.15.172:0/2694106831 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a2affd700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a2c103530 0x7f0a2c102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 -- 172.21.15.172:0/2694106831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a2c103530 msgr2=0x7f0a2c102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a2c103530 0x7f0a2c102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 -- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c108790 msgr2=0x7f0a2c101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:18.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c108790 0x7f0a2c101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 -- 172.21.15.172:0/2694106831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c1022d0 con 0x7f0a2c105e60 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.220+0000 7f0a227fc700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 0x7f0a2c1014e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b530 tx=0x7f0a1400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.221+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14015020 con 0x7f0a2c105e60 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.221+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a14011ce0 con 0x7f0a2c105e60 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.221+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c1a2e30 con 0x7f0a2c105e60 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.221+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14010e40 con 0x7f0a2c105e60 2024-04-23T18:13:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.221+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c1a3360 con 0x7f0a2c105e60 2024-04-23T18:13:18.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.222+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c069730 con 0x7f0a2c105e60 2024-04-23T18:13:18.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.223+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0a14018020 con 0x7f0a2c105e60 2024-04-23T18:13:18.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.224+0000 7f0a28ff9700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0a180a3c30 0x7f0a180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:18.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.224+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f0a140ce610 con 0x7f0a2c105e60 2024-04-23T18:13:18.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.224+0000 7f0a2affd700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0a180a3c30 0x7f0a180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:13:18.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.225+0000 7f0a2affd700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0a180a3c30 0x7f0a180a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009550 tx=0x7f0a1c0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:13:18.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.228+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a1401f020 con 0x7f0a2c105e60 2024-04-23T18:13:18.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:17 smithi173 ceph-mon[32906]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:13:18.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:17 smithi037 ceph-mon[29404]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:13:18.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:18.435+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0a2c062e80 con 0x7f0a180a3c30 2024-04-23T18:13:19.202 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:18 smithi172 ceph-mon[32456]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:19.202 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:18 smithi172 ceph-mon[32456]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:13:19.202 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:13:19.202 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:13:19.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:19.257 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:18 smithi173 ceph-mon[32906]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:19.257 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:18 smithi173 ceph-mon[32906]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:13:19.257 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:13:19.257 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:13:19.257 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:19.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:18 smithi037 ceph-mon[29404]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:19.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:18 smithi037 ceph-mon[29404]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:13:19.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:13:19.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:13:19.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:21.218 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:20 smithi172 ceph-mon[32456]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:21.218 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:20 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/2997902692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]: dispatch 2024-04-23T18:13:21.218 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:20 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]: dispatch 2024-04-23T18:13:21.219 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:20 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]': finished 2024-04-23T18:13:21.219 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:20 smithi172 ceph-mon[32456]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:13:21.219 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:20 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:21.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:20 smithi173 ceph-mon[32906]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:21.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:20 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/2997902692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]: dispatch 2024-04-23T18:13:21.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:20 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]: dispatch 2024-04-23T18:13:21.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:20 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]': finished 2024-04-23T18:13:21.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:20 smithi173 ceph-mon[32906]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:13:21.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:20 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:20 smithi037 ceph-mon[29404]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:20 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/2997902692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]: dispatch 2024-04-23T18:13:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:20 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]: dispatch 2024-04-23T18:13:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:20 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28317ec5-ba88-452e-95ba-a312941588bc"}]': finished 2024-04-23T18:13:21.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:20 smithi037 ceph-mon[29404]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:13:21.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:20 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:22.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:21 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/1259711438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:13:22.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:21 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/1259711438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:13:22.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:21 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/1259711438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:13:23.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:22 smithi037 ceph-mon[29404]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:23.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:22 smithi172 ceph-mon[32456]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:23.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:22 smithi173 ceph-mon[32906]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:25.080 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:24 smithi172 ceph-mon[32456]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:25.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:24 smithi173 ceph-mon[32906]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:25.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:24 smithi037 ceph-mon[29404]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:27.201 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:26 smithi172 ceph-mon[32456]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:26 smithi173 ceph-mon[32906]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:26 smithi037 ceph-mon[29404]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:28 smithi172 ceph-mon[32456]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:28 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:13:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:28 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:28 smithi172 ceph-mon[32456]: Deploying daemon osd.1 on smithi172 2024-04-23T18:13:29.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:28 smithi173 ceph-mon[32906]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:29.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:13:29.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:29.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:28 smithi173 ceph-mon[32906]: Deploying daemon osd.1 on smithi172 2024-04-23T18:13:29.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:28 smithi037 ceph-mon[29404]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:29.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:13:29.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:29.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:28 smithi037 ceph-mon[29404]: Deploying daemon osd.1 on smithi172 2024-04-23T18:13:31.130 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:30 smithi172 ceph-mon[32456]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:30 smithi173 ceph-mon[32906]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:30 smithi037 ceph-mon[29404]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:33.192 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:32 smithi172 ceph-mon[32456]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:32 smithi173 ceph-mon[32906]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:32 smithi037 ceph-mon[29404]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:34.786 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:34.786 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:34.786 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:34.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:34.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:34.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:35.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:35 smithi172 ceph-mon[32456]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:35.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:35.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:35.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:35 smithi173 ceph-mon[32906]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:35.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:35.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:35.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:35 smithi037 ceph-mon[29404]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.055+0000 7f0a28ff9700 1 -- 172.21.15.172:0/2694106831 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0a2c062e80 con 0x7f0a180a3c30 2024-04-23T18:13:37.050 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 1 on host 'smithi172' 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0a180a3c30 msgr2=0x7f0a180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0a180a3c30 0x7f0a180a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009550 tx=0x7f0a1c0091b0 comp rx=0 tx=0).stop 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 msgr2=0x7f0a2c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 0x7f0a2c1014e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400b530 tx=0x7f0a1400b840 comp rx=0 tx=0).stop 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 shutdown_connections 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0a180a3c30 0x7f0a180a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c108790 0x7f0a2c101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a2c103530 0x7f0a2c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 --2- 172.21.15.172:0/2694106831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a2c105e60 0x7f0a2c1014e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 >> 172.21.15.172:0/2694106831 conn(0x7f0a2c0fcf40 msgr2=0x7f0a2c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:13:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 shutdown_connections 2024-04-23T18:13:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:13:37.058+0000 7f0a314a0700 1 -- 172.21.15.172:0/2694106831 wait complete. 2024-04-23T18:13:37.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:37 smithi172 ceph-mon[32456]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:37.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:37 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:37 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:37 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:37.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:37 smithi172 ceph-mon[32456]: from='osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:13:37.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:37 smithi172 ceph-mon[32456]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:13:37.676 DEBUG:teuthology.orchestra.run.smithi172:osd.1> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T18:13:37.678 INFO:tasks.cephadm:Deploying osd.2 on smithi173 with /dev/vg_nvme/lv_4... 2024-04-23T18:13:37.678 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:13:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:37 smithi173 ceph-mon[32906]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:37 smithi173 ceph-mon[32906]: from='osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:13:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:37 smithi173 ceph-mon[32906]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:13:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:37 smithi037 ceph-mon[29404]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:37 smithi037 ceph-mon[29404]: from='osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:13:37.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:37 smithi037 ceph-mon[29404]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:13:37.921 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:13:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:39 smithi173 ceph-mon[32906]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:39 smithi173 ceph-mon[32906]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:13:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:39 smithi173 ceph-mon[32906]: from='osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:13:39.163 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:39 smithi173 ceph-mon[32906]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:13:39.164 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:39 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:39.164 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:39 smithi173 ceph-mon[32906]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:13:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-mon[32456]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-mon[32456]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:13:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-mon[32456]: from='osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:13:39.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-mon[32456]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:13:39.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:39.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-mon[32456]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:13:39.339 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1[37033]: 2024-04-23T18:13:39.065+0000 7f9441115700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T18:13:39.339 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:13:39 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1[37033]: 2024-04-23T18:13:39.068+0000 7f943cf08700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:13:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:39 smithi037 ceph-mon[29404]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:39 smithi037 ceph-mon[29404]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:13:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:39 smithi037 ceph-mon[29404]: from='osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:13:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:39 smithi037 ceph-mon[29404]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:13:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:39 smithi037 ceph-mon[29404]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:13:39.712 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:13:39.712 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:13:39.712 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T18:13:39.712 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T18:13:39.712 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112216 s, 934 MB/s 2024-04-23T18:13:39.712 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T18:13:40.286 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:40 smithi172 ceph-mon[32456]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:13:40.287 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:40 smithi172 ceph-mon[32456]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:13:40.287 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:40.287 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:40.351 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:40 smithi173 ceph-mon[32906]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:13:40.351 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:40 smithi173 ceph-mon[32906]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:13:40.351 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:40.351 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:40 smithi037 ceph-mon[29404]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:13:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:40 smithi037 ceph-mon[29404]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:13:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:40.381 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-04-23T18:13:40.626 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:13:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:13:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:13:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] boot 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi172 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi172 to 17088M 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:41.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] boot 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi172 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:13:41.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi172 to 17088M 2024-04-23T18:13:41.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:41.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:41.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:13:41.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:13:41.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:13:41.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: osd.1 [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] boot 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi172 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi172 to 17088M 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:13:41.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:42.232 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 -- 172.21.15.173:0/3093480407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1014f0 msgr2=0x7f23fc10b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:42.232 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 --2- 172.21.15.173:0/3093480407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1014f0 0x7f23fc10b0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b0a0 tx=0x7f23f400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:13:42.232 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 -- 172.21.15.173:0/3093480407 shutdown_connections 2024-04-23T18:13:42.232 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 --2- 172.21.15.173:0/3093480407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1014f0 0x7f23fc10b0a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 --2- 172.21.15.173:0/3093480407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1040b0 0x7f23fc1044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 --2- 172.21.15.173:0/3093480407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23fc100b30 0x7f23fc100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 -- 172.21.15.173:0/3093480407 >> 172.21.15.173:0/3093480407 conn(0x7f23fc0fc6b0 msgr2=0x7f23fc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 -- 172.21.15.173:0/3093480407 shutdown_connections 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 -- 172.21.15.173:0/3093480407 wait complete. 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 Processor -- start 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.236+0000 7f2403856700 1 -- start start 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23fc100b30 0x7f23fc198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1014f0 0x7f23fc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 0x7f23fc1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10e680 con 0x7f23fc100b30 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10e500 con 0x7f23fc1014f0 2024-04-23T18:13:42.233 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10e800 con 0x7f23fc1040b0 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 0x7f23fc1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 0x7f23fc1a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50736/0 (socket says 172.21.15.173:50736) 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 -- 172.21.15.173:0/791222219 learned_addr learned my addr 172.21.15.173:0/791222219 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2400df1700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1014f0 0x7f23fc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 -- 172.21.15.173:0/791222219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1014f0 msgr2=0x7f23fc199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1014f0 0x7f23fc199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 -- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23fc100b30 msgr2=0x7f23fc198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23fc100b30 0x7f23fc198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 -- 172.21.15.173:0/791222219 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc1a0960 con 0x7f23fc1040b0 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2400df1700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1014f0 0x7f23fc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2401df3700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 0x7f23fc1a0290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b070 tx=0x7f23f4004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:13:42.234 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4021070 con 0x7f23fc1040b0 2024-04-23T18:13:42.235 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f4004020 con 0x7f23fc1040b0 2024-04-23T18:13:42.235 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1a0bf0 con 0x7f23fc1040b0 2024-04-23T18:13:42.235 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f401d690 con 0x7f23fc1040b0 2024-04-23T18:13:42.235 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.237+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a10f0 con 0x7f23fc1040b0 2024-04-23T18:13:42.235 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.238+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23fc069730 con 0x7f23fc1040b0 2024-04-23T18:13:42.236 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.240+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f23f40044f0 con 0x7f23fc1040b0 2024-04-23T18:13:42.236 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.240+0000 7f23f27fc700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23e80a3c30 0x7f23e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:13:42.236 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.240+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f23f400f020 con 0x7f23fc1040b0 2024-04-23T18:13:42.236 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.240+0000 7f24015f2700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23e80a3c30 0x7f23e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:13:42.236 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.241+0000 7f24015f2700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23e80a3c30 0x7f23e80a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23fc19a1f0 tx=0x7f23f8006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:13:42.238 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.244+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23f402a020 con 0x7f23fc1040b0 2024-04-23T18:13:42.446 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:13:42.451+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f23fc062e80 con 0x7f23e80a3c30 2024-04-23T18:13:42.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:42 smithi173 ceph-mon[32906]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:13:42.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:42 smithi037 ceph-mon[29404]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:13:42.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:42 smithi172 ceph-mon[32456]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:13:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:43 smithi173 ceph-mon[32906]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:43 smithi173 ceph-mon[32906]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:13:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:13:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:13:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:43 smithi037 ceph-mon[29404]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:43 smithi037 ceph-mon[29404]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:13:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:13:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:13:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:43.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:43 smithi172 ceph-mon[32456]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:43.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:43 smithi172 ceph-mon[32456]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:13:43.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:13:43.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:13:43.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:45.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:45 smithi037 ceph-mon[29404]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:45.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:45 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/727589776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]: dispatch 2024-04-23T18:13:45.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:45 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]: dispatch 2024-04-23T18:13:45.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:45 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]': finished 2024-04-23T18:13:45.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:45 smithi037 ceph-mon[29404]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:13:45.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:45 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:13:45.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:45 smithi172 ceph-mon[32456]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:45.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:45 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/727589776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]: dispatch 2024-04-23T18:13:45.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:45 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]: dispatch 2024-04-23T18:13:45.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:45 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]': finished 2024-04-23T18:13:45.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:45 smithi172 ceph-mon[32456]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:13:45.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:45 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:13:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:45 smithi173 ceph-mon[32906]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:45 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/727589776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]: dispatch 2024-04-23T18:13:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:45 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]: dispatch 2024-04-23T18:13:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:45 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00f910a1-90d1-485c-a253-d0028ce0cc8d"}]': finished 2024-04-23T18:13:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:45 smithi173 ceph-mon[32906]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:13:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:45 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:13:46.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:46 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/1413449469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:13:46.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:46 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/1413449469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:13:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:46 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/1413449469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:13:47.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:47 smithi037 ceph-mon[29404]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:47.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:47 smithi172 ceph-mon[32456]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:47.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:47 smithi173 ceph-mon[32906]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:49.533 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:49 smithi173 ceph-mon[32906]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:49.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:49 smithi037 ceph-mon[29404]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:49.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:49 smithi172 ceph-mon[32456]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:51.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:51 smithi173 ceph-mon[32906]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:51.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:51 smithi172 ceph-mon[32456]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:51 smithi037 ceph-mon[29404]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:52.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:13:52.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:52.826 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:13:52.826 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:13:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:13:53.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:53 smithi173 ceph-mon[32906]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:53.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:53 smithi173 ceph-mon[32906]: Deploying daemon osd.2 on smithi173 2024-04-23T18:13:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:53 smithi172 ceph-mon[32456]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:53 smithi172 ceph-mon[32456]: Deploying daemon osd.2 on smithi173 2024-04-23T18:13:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:53 smithi037 ceph-mon[29404]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:53 smithi037 ceph-mon[29404]: Deploying daemon osd.2 on smithi173 2024-04-23T18:13:55.810 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:55 smithi173 ceph-mon[32906]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:55 smithi172 ceph-mon[32456]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:55 smithi037 ceph-mon[29404]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:57.577 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:57 smithi173 ceph-mon[32906]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:57.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:57 smithi172 ceph-mon[32456]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:57.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:57 smithi037 ceph-mon[29404]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:13:58.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:58.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:58.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:59.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:59.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:59.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:13:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:13:59.854 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:13:59 smithi173 ceph-mon[32906]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:00.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:13:59 smithi037 ceph-mon[29404]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:00.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:13:59 smithi172 ceph-mon[32456]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:00.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:00 smithi173 ceph-mon[32906]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:01.013 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 2 on host 'smithi173' 2024-04-23T18:14:01.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.018+0000 7f23f27fc700 1 -- 172.21.15.173:0/791222219 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f23fc062e80 con 0x7f23e80a3c30 2024-04-23T18:14:01.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23e80a3c30 msgr2=0x7f23e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:01.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23e80a3c30 0x7f23e80a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23fc19a1f0 tx=0x7f23f8006d20 comp rx=0 tx=0).stop 2024-04-23T18:14:01.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 msgr2=0x7f23fc1a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:01.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 0x7f23fc1a0290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b070 tx=0x7f23f4004ef0 comp rx=0 tx=0).stop 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 shutdown_connections 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23e80a3c30 0x7f23e80a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23fc100b30 0x7f23fc198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23fc1040b0 0x7f23fc1a0290 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 --2- 172.21.15.173:0/791222219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc1014f0 0x7f23fc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 >> 172.21.15.173:0/791222219 conn(0x7f23fc0fc6b0 msgr2=0x7f23fc107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 shutdown_connections 2024-04-23T18:14:01.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:01.021+0000 7f2403856700 1 -- 172.21.15.173:0/791222219 wait complete. 2024-04-23T18:14:01.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:00 smithi172 ceph-mon[32456]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:01.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:00 smithi037 ceph-mon[29404]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:01.623 DEBUG:teuthology.orchestra.run.smithi173:osd.2> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T18:14:01.626 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-04-23T18:14:01.626 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:14:01.882 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:14:02.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:01 smithi037 ceph-mon[29404]: from='osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:14:02.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:01 smithi037 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:14:02.330 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.331 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.331 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.331 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.331 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:01 smithi172 ceph-mon[32456]: from='osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:14:02.331 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:01 smithi172 ceph-mon[32456]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:14:02.336 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.336 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.336 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.336 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:02.336 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:01 smithi173 ceph-mon[32906]: from='osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:14:02.336 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:01 smithi173 ceph-mon[32906]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:14:03.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:03 smithi037 ceph-mon[29404]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:03.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:03 smithi037 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:14:03.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:03 smithi037 ceph-mon[29404]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:14:03.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:03 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:03.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:03 smithi037 ceph-mon[29404]: from='osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:14:03.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:03 smithi037 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:14:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:03 smithi172 ceph-mon[32456]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:03 smithi172 ceph-mon[32456]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:14:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:03 smithi172 ceph-mon[32456]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:14:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:03 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:03 smithi172 ceph-mon[32456]: from='osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:14:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:03 smithi172 ceph-mon[32456]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:14:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-mon[32906]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-mon[32906]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:14:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-mon[32906]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:14:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-mon[32906]: from='osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:14:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-mon[32906]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:14:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:14:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:14:03.983 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-23T18:14:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-23T18:14:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111689 s, 939 MB/s 2024-04-23T18:14:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-23T18:14:04.235 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-mon[32906]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:14:04.235 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-mon[32906]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:14:04.235 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.235 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.235 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-mon[32906]: from='osd.2 ' entity='osd.2' 2024-04-23T18:14:04.236 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.236 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2[38068]: 2024-04-23T18:14:03.986+0000 7fe1c2346700 -1 osd.2 0 waiting for initial osdmap 2024-04-23T18:14:04.236 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2[38068]: 2024-04-23T18:14:04.137+0000 7fe1ba131700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:14:04.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:04 smithi037 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:14:04.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:04 smithi037 ceph-mon[29404]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:14:04.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:04 smithi037 ceph-mon[29404]: from='osd.2 ' entity='osd.2' 2024-04-23T18:14:04.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:04 smithi172 ceph-mon[32456]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:14:04.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:04 smithi172 ceph-mon[32456]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:14:04.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:04 smithi172 ceph-mon[32456]: from='osd.2 ' entity='osd.2' 2024-04-23T18:14:04.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:04.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-04-23T18:14:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi173 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-23T18:14:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:14:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:14:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi173 2024-04-23T18:14:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi173 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:05.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:06.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:06 smithi037 ceph-mon[29404]: osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] boot 2024-04-23T18:14:06.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:06 smithi037 ceph-mon[29404]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:14:06.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:06 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:06.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:06 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:14:06.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:06 smithi173 ceph-mon[32906]: osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] boot 2024-04-23T18:14:06.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:06 smithi173 ceph-mon[32906]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:14:06.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:06 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:06.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:06 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 -- 172.21.15.37:0/4288566562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec601019a0 msgr2=0x7fec60101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 --2- 172.21.15.37:0/4288566562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec601019a0 0x7fec60101dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec500099c0 tx=0x7fec50009cd0 comp rx=0 tx=0).stop 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 -- 172.21.15.37:0/4288566562 shutdown_connections 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 --2- 172.21.15.37:0/4288566562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec601019a0 0x7fec60101dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 --2- 172.21.15.37:0/4288566562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec60103540 0x7fec60109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 --2- 172.21.15.37:0/4288566562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec60102ba0 0x7fec60103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 -- 172.21.15.37:0/4288566562 >> 172.21.15.37:0/4288566562 conn(0x7fec600fcf40 msgr2=0x7fec600ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 -- 172.21.15.37:0/4288566562 shutdown_connections 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 -- 172.21.15.37:0/4288566562 wait complete. 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.438+0000 7fec660fe700 1 Processor -- start 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 -- start start 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec601019a0 0x7fec6019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 0x7fec6019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec60103540 0x7fec601a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec6010d430 con 0x7fec60102ba0 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec6010d2b0 con 0x7fec60103540 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec660fe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fec6010d5b0 con 0x7fec601019a0 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 0x7fec6019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 0x7fec6019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60838/0 (socket says 172.21.15.37:60838) 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 -- 172.21.15.37:0/3747725725 learned_addr learned my addr 172.21.15.37:0/3747725725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5f7fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec601019a0 0x7fec6019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:06.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5ffff700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec60103540 0x7fec601a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 -- 172.21.15.37:0/3747725725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec601019a0 msgr2=0x7fec6019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec601019a0 0x7fec6019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 -- 172.21.15.37:0/3747725725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec60103540 msgr2=0x7fec601a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec60103540 0x7fec601a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 -- 172.21.15.37:0/3747725725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec601a2a70 con 0x7fec60102ba0 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.439+0000 7fec5effd700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 0x7fec6019b350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec5400d980 tx=0x7fec5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:14:06.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.440+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54016070 con 0x7fec60102ba0 2024-04-23T18:14:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.440+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec54009d80 con 0x7fec60102ba0 2024-04-23T18:14:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.440+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec601a2d60 con 0x7fec60102ba0 2024-04-23T18:14:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.440+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5400f3c0 con 0x7fec60102ba0 2024-04-23T18:14:06.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.440+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec601a3240 con 0x7fec60102ba0 2024-04-23T18:14:06.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.441+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec60069760 con 0x7fec60102ba0 2024-04-23T18:14:06.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.442+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fec54019070 con 0x7fec60102ba0 2024-04-23T18:14:06.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.443+0000 7fec5cff9700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec480a3bb0 0x7fec480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:06.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.443+0000 7fec5f7fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec480a3bb0 0x7fec480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:06.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.443+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fec540ce990 con 0x7fec60102ba0 2024-04-23T18:14:06.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.443+0000 7fec5f7fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec480a3bb0 0x7fec480a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec50006010 tx=0x7fec500036a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:14:06.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.448+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec60069760 con 0x7fec60102ba0 2024-04-23T18:14:06.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:06 smithi172 ceph-mon[32456]: osd.2 [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] boot 2024-04-23T18:14:06.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:06 smithi172 ceph-mon[32456]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:14:06.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:06 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:14:06.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:06 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:14:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:06.665+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fec600ff190 con 0x7fec480a3bb0 2024-04-23T18:14:07.352 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:07.352 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:14:07.352 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:14:07.352 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:14:07.352 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:07.352 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:07.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:07.353 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41823]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:14:07.353 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.353 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41823]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.353 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41823]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:14:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:14:07.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:14:07.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:07.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:07.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46386]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46386]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46386]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46389]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.372 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46389]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.470 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:14:07.470 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:14:07.470 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:14:07.470 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:14:07.471 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:07.471 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:07.471 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:07.471 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40991]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:14:07.471 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40991]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.472 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40991]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.472 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40991]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.472 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40994]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:14:07.472 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40994]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.472 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40994]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.472 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40994]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41829]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41829]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41832]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41832]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41832]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41832]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.607 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:14:07.607 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.607 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41826]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.607 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:14:07 smithi173 sudo[41826]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:14:07.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46392]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46392]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46395]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:14:07.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46395]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:07 smithi037 sudo[46395]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40997]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[40997]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[41000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[41000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:14:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[41000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:14:07.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:07 smithi172 sudo[41000]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:14:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:14:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:14:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:08 smithi037 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:08.408+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec540987a0 con 0x7fec60102ba0 2024-04-23T18:14:08.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:08.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:08 smithi172 ceph-mon[32456]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:14:08.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:08 smithi173 ceph-mon[32906]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:14:09.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:09 smithi037 ceph-mon[29404]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:09.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:09 smithi037 ceph-mon[29404]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:14:09.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:09 smithi037 ceph-mon[29404]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:14:09.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:09 smithi172 ceph-mon[32456]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:09.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:09 smithi172 ceph-mon[32456]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:14:09.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:09 smithi172 ceph-mon[32456]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:14:09.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:09 smithi173 ceph-mon[32906]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:09.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:09 smithi173 ceph-mon[32906]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:14:09.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:09 smithi173 ceph-mon[32906]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:14:10.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:10 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2560489498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6533c224-ca86-4499-9182-283a32bb2774"}]: dispatch 2024-04-23T18:14:10.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:10 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2560489498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6533c224-ca86-4499-9182-283a32bb2774"}]': finished 2024-04-23T18:14:10.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:10 smithi037 ceph-mon[29404]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:14:10.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:10.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:10 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2560489498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6533c224-ca86-4499-9182-283a32bb2774"}]: dispatch 2024-04-23T18:14:10.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:10 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2560489498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6533c224-ca86-4499-9182-283a32bb2774"}]': finished 2024-04-23T18:14:10.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:10 smithi172 ceph-mon[32456]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:14:10.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:10.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:10 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2560489498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6533c224-ca86-4499-9182-283a32bb2774"}]: dispatch 2024-04-23T18:14:10.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:10 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2560489498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6533c224-ca86-4499-9182-283a32bb2774"}]': finished 2024-04-23T18:14:10.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:10 smithi173 ceph-mon[32906]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:14:10.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:11.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:11 smithi037 ceph-mon[29404]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:11.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:11 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/127865474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:14:11.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:11 smithi172 ceph-mon[32456]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:11.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:11 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/127865474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:14:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:11 smithi173 ceph-mon[32906]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:11.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:11 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/127865474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:14:13.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:13 smithi037 ceph-mon[29404]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:13.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:13 smithi172 ceph-mon[32456]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:13.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:13 smithi173 ceph-mon[32906]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:15.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:15 smithi037 ceph-mon[29404]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:15.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:15 smithi172 ceph-mon[32456]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:15.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:15 smithi173 ceph-mon[32906]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:17.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:17 smithi037 ceph-mon[29404]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:17.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:17 smithi172 ceph-mon[32456]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:17.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:17 smithi173 ceph-mon[32906]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:18.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:14:18.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:18.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:18 smithi172 ceph-mon[32456]: Deploying daemon osd.3 on smithi037 2024-04-23T18:14:18.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:14:18.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:18.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:18 smithi173 ceph-mon[32906]: Deploying daemon osd.3 on smithi037 2024-04-23T18:14:18.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:14:18.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:18.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:18 smithi037 ceph-mon[29404]: Deploying daemon osd.3 on smithi037 2024-04-23T18:14:19.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:19 smithi037 ceph-mon[29404]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:19.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:19 smithi173 ceph-mon[32906]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:19.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:19 smithi172 ceph-mon[32456]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:21.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:21 smithi037 ceph-mon[29404]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:21.390 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:21 smithi172 ceph-mon[32456]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:21.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:21 smithi173 ceph-mon[32906]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:23.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:23 smithi037 ceph-mon[29404]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:23.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:23.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:23.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:23.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:23 smithi172 ceph-mon[32456]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:23.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:23.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:23.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:23 smithi173 ceph-mon[32906]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:25.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:25 smithi037 ceph-mon[29404]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:25.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:25 smithi172 ceph-mon[32456]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:25.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:25 smithi173 ceph-mon[32906]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:26.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:26.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:26.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.421+0000 7fec5cff9700 1 -- 172.21.15.37:0/3747725725 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec600ff190 con 0x7fec480a3bb0 2024-04-23T18:14:26.417 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec480a3bb0 msgr2=0x7fec480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec480a3bb0 0x7fec480a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec50006010 tx=0x7fec500036a0 comp rx=0 tx=0).stop 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 msgr2=0x7fec6019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 0x7fec6019b350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fec5400d980 tx=0x7fec5400dc90 comp rx=0 tx=0).stop 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 shutdown_connections 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec480a3bb0 0x7fec480a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec60103540 0x7fec601a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec601019a0 0x7fec6019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 --2- 172.21.15.37:0/3747725725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec60102ba0 0x7fec6019b350 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:26.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 >> 172.21.15.37:0/3747725725 conn(0x7fec600fcf40 msgr2=0x7fec600fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:14:26.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 shutdown_connections 2024-04-23T18:14:26.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:14:26.424+0000 7fec660fe700 1 -- 172.21.15.37:0/3747725725 wait complete. 2024-04-23T18:14:26.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:26.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:26.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:26.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:26.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:26.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.100 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T18:14:27.103 INFO:tasks.cephadm:Deploying osd.4 on smithi172 with /dev/vg_nvme/lv_3... 2024-04-23T18:14:27.103 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:14:27.350 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:14:27.660 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:27 smithi172 ceph-mon[32456]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:27.660 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:27 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.660 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:27 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.660 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:27 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:27.660 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:27 smithi172 ceph-mon[32456]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:14:27.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:27 smithi173 ceph-mon[32906]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:27.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:27 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:27 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:27 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:27.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:27 smithi173 ceph-mon[32906]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:14:27.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:27 smithi037 ceph-mon[29404]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:27.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:27 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:27 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:27.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:27 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:27.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:27 smithi037 ceph-mon[29404]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:14:28.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:28 smithi037 ceph-mon[29404]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:14:28.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:28 smithi037 ceph-mon[29404]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:14:28.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:28 smithi037 ceph-mon[29404]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:14:28.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:28.442 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:14:28 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:14:28.418+0000 7f88afcc2700 -1 osd.3 0 waiting for initial osdmap 2024-04-23T18:14:28.442 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:14:28 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:14:28.425+0000 7f88abab5700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:14:28.713 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:28 smithi172 ceph-mon[32456]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:14:28.713 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:28 smithi172 ceph-mon[32456]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:14:28.714 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:28 smithi172 ceph-mon[32456]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:14:28.714 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:28 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:28.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:28 smithi173 ceph-mon[32906]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:14:28.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:28 smithi173 ceph-mon[32906]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:14:28.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:28 smithi173 ceph-mon[32906]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:14:28.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.299 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:14:29.300 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:14:29.300 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-23T18:14:29.300 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-23T18:14:29.300 INFO:teuthology.orchestra.run.smithi172.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114789 s, 913 MB/s 2024-04-23T18:14:29.300 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 2024-04-23T18:14:29.300 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-23T18:14:29.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:29 smithi037 ceph-mon[29404]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:29.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:29 smithi037 ceph-mon[29404]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:14:29.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:29 smithi037 ceph-mon[29404]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:14:29.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:29 smithi172 ceph-mon[32456]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:29 smithi172 ceph-mon[32456]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:14:29.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:29 smithi172 ceph-mon[32456]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:14:29.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:29 smithi173 ceph-mon[32906]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:14:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:29 smithi173 ceph-mon[32906]: from='osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:14:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:29 smithi173 ceph-mon[32906]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:14:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:29.889 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-04-23T18:14:30.133 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] boot 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.539 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:30.540 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:30.540 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.540 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:30.540 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:30.540 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] boot 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:30.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:30.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:30.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:30.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:14:30.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:14:30.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: osd.3 [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] boot 2024-04-23T18:14:30.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:14:30.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:30.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:31 smithi172 ceph-mon[32456]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:31 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi037 2024-04-23T18:14:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:31 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-23T18:14:31.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:31 smithi172 ceph-mon[32456]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:14:31.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2053548538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd98104560 msgr2=0x7fbd9810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:31.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2053548538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd98104560 0x7fbd9810ae10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009fc0 tx=0x7fbd9000b330 comp rx=0 tx=0).stop 2024-04-23T18:14:31.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2053548538 shutdown_connections 2024-04-23T18:14:31.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2053548538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd98104560 0x7fbd9810ae10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:31.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2053548538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd981029c0 0x7fbd98102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:31.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2053548538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 0x7fbd98104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:31.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2053548538 >> 172.21.15.172:0/2053548538 conn(0x7fbd980fdf80 msgr2=0x7fbd981003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:14:31.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2053548538 shutdown_connections 2024-04-23T18:14:31.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.629+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2053548538 wait complete. 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 Processor -- start 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 -- start start 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 0x7fbd980767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 0x7fbd98074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd98104560 0x7fbd98075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810e460 con 0x7fbd98103bc0 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810e2e0 con 0x7fbd981029c0 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9e79f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810e5e0 con 0x7fbd98104560 2024-04-23T18:14:31.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9d79d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 0x7fbd980767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9d79d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 0x7fbd980767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58892/0 (socket says 172.21.15.172:58892) 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9cf9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 0x7fbd98074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.630+0000 7fbd9d79d700 1 -- 172.21.15.172:0/2549962449 learned_addr learned my addr 172.21.15.172:0/2549962449 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9df9e700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd98104560 0x7fbd98075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9d79d700 1 -- 172.21.15.172:0/2549962449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd98104560 msgr2=0x7fbd98075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9d79d700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd98104560 0x7fbd98075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9d79d700 1 -- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 msgr2=0x7fbd98074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9d79d700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 0x7fbd98074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9d79d700 1 -- 172.21.15.172:0/2549962449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd98075c20 con 0x7fbd981029c0 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9cf9c700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 0x7fbd98074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9d79d700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 0x7fbd980767e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800d980 tx=0x7fbd8800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88017070 con 0x7fbd981029c0 2024-04-23T18:14:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd88010460 con 0x7fbd981029c0 2024-04-23T18:14:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800de60 con 0x7fbd981029c0 2024-04-23T18:14:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981a2c00 con 0x7fbd981029c0 2024-04-23T18:14:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.631+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a3110 con 0x7fbd981029c0 2024-04-23T18:14:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.632+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd98069730 con 0x7fbd981029c0 2024-04-23T18:14:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.633+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd88010610 con 0x7fbd981029c0 2024-04-23T18:14:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.634+0000 7fbd8e7fc700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd840a3cc0 0x7fbd840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:31.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.634+0000 7fbd9cf9c700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd840a3cc0 0x7fbd840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:31.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.634+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fbd880ced00 con 0x7fbd981029c0 2024-04-23T18:14:31.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.635+0000 7fbd9cf9c700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd840a3cc0 0x7fbd840a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005fd0 tx=0x7fbd94009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:14:31.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.639+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd8809c8e0 con 0x7fbd981029c0 2024-04-23T18:14:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:31 smithi037 ceph-mon[29404]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:31 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi037 2024-04-23T18:14:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:31 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-23T18:14:31.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:31 smithi037 ceph-mon[29404]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:14:31.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:31.844+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbd98062f40 con 0x7fbd840a3cc0 2024-04-23T18:14:31.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:31 smithi173 ceph-mon[32906]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:31.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:31 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi037 2024-04-23T18:14:31.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:31 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-23T18:14:31.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:31 smithi173 ceph-mon[32906]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:14:32.689 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:32 smithi172 ceph-mon[32456]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:14:32.689 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:32 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:32.689 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:32 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:32.689 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:32 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:32.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:32 smithi173 ceph-mon[32906]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:14:32.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:32 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:32.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:32 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:32.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:32 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:32.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:32 smithi037 ceph-mon[29404]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:14:32.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:32 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:32.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:32 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:32.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:32 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:33.686 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:33 smithi172 ceph-mon[32456]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:33.687 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:33 smithi172 ceph-mon[32456]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:33.687 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:33 smithi172 ceph-mon[32456]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:14:33.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:33 smithi173 ceph-mon[32906]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:33.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:33 smithi173 ceph-mon[32906]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:33.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:33 smithi173 ceph-mon[32906]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:14:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:33 smithi037 ceph-mon[29404]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:33 smithi037 ceph-mon[29404]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:33 smithi037 ceph-mon[29404]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:14:34.764 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:34 smithi172 ceph-mon[32456]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:14:34.765 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:34 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/2457352940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]: dispatch 2024-04-23T18:14:34.765 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:34 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]: dispatch 2024-04-23T18:14:34.765 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:34 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]': finished 2024-04-23T18:14:34.765 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:34 smithi172 ceph-mon[32456]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:14:34.765 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:34.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:34 smithi173 ceph-mon[32906]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:14:34.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:34 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/2457352940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]: dispatch 2024-04-23T18:14:34.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:34 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]: dispatch 2024-04-23T18:14:34.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:34 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]': finished 2024-04-23T18:14:34.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:34 smithi173 ceph-mon[32906]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:14:34.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:34 smithi037 ceph-mon[29404]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:14:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/2457352940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]: dispatch 2024-04-23T18:14:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:34 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]: dispatch 2024-04-23T18:14:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:34 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e24984b-edbd-4d4f-8716-6713ce2a039c"}]': finished 2024-04-23T18:14:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:34 smithi037 ceph-mon[29404]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:14:34.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:35.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:35 smithi172 ceph-mon[32456]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:35.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/727999585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:14:35.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:35 smithi173 ceph-mon[32906]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:35.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/727999585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:14:35.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:35 smithi037 ceph-mon[29404]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:35.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/727999585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:14:37.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:37 smithi172 ceph-mon[32456]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:37.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:37 smithi173 ceph-mon[32906]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:37 smithi037 ceph-mon[29404]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:39.769 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:39 smithi172 ceph-mon[32456]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:14:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:39 smithi173 ceph-mon[32906]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:14:39.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:39 smithi037 ceph-mon[29404]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:14:41.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:41 smithi172 ceph-mon[32456]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-23T18:14:41.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:41 smithi173 ceph-mon[32906]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-23T18:14:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:41 smithi037 ceph-mon[29404]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-23T18:14:42.698 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:42 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:14:42.699 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:42 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:42.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:42 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:14:42.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:42 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:42.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:42 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:14:42.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:42 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:43.729 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:43 smithi172 ceph-mon[32456]: Deploying daemon osd.4 on smithi172 2024-04-23T18:14:43.729 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:43 smithi172 ceph-mon[32456]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-23T18:14:43.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:43 smithi173 ceph-mon[32906]: Deploying daemon osd.4 on smithi172 2024-04-23T18:14:43.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:43 smithi173 ceph-mon[32906]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-23T18:14:43.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:43 smithi037 ceph-mon[29404]: Deploying daemon osd.4 on smithi172 2024-04-23T18:14:43.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:43 smithi037 ceph-mon[29404]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-23T18:14:45.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:45 smithi172 ceph-mon[32456]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:14:45.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:45 smithi173 ceph-mon[32906]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:14:45.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:45 smithi037 ceph-mon[29404]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:14:47.819 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:47 smithi172 ceph-mon[32456]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-23T18:14:47.819 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:47 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:47.819 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:47 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:47.819 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:47 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:47.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:47 smithi173 ceph-mon[32906]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-23T18:14:47.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:47 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:47.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:47 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:47.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:47 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:47.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:47 smithi037 ceph-mon[29404]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-23T18:14:47.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:47 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:47.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:47 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:47.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:47 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:49.694 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:49 smithi172 ceph-mon[32456]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:14:49.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:49 smithi173 ceph-mon[32906]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:14:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:49 smithi037 ceph-mon[29404]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:14:50.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.459+0000 7fbd8e7fc700 1 -- 172.21.15.172:0/2549962449 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd98062f40 con 0x7fbd840a3cc0 2024-04-23T18:14:50.453 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 4 on host 'smithi172' 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd840a3cc0 msgr2=0x7fbd840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd840a3cc0 0x7fbd840a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005fd0 tx=0x7fbd94009310 comp rx=0 tx=0).stop 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 msgr2=0x7fbd980767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 0x7fbd980767e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800d980 tx=0x7fbd8800dd40 comp rx=0 tx=0).stop 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 shutdown_connections 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd840a3cc0 0x7fbd840a6180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd98104560 0x7fbd98075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd981029c0 0x7fbd980767e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 --2- 172.21.15.172:0/2549962449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd98103bc0 0x7fbd98074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 >> 172.21.15.172:0/2549962449 conn(0x7fbd980fdf80 msgr2=0x7fbd98106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 shutdown_connections 2024-04-23T18:14:50.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:14:50.462+0000 7fbd9e79f700 1 -- 172.21.15.172:0/2549962449 wait complete. 2024-04-23T18:14:51.076 DEBUG:teuthology.orchestra.run.smithi172:osd.4> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T18:14:51.080 INFO:tasks.cephadm:Deploying osd.5 on smithi173 with /dev/vg_nvme/lv_3... 2024-04-23T18:14:51.080 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:14:51.323 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:14:51.681 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:51.681 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.681 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.681 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: from='osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:14:51.681 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:14:51.682 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.682 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:51 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.798 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:51.798 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.798 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.798 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: from='osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:14:51.798 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:14:51.799 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.799 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:51 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: from='osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:14:51.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:14:51.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:51.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:51 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:52 smithi172 ceph-mon[32456]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:14:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:52 smithi172 ceph-mon[32456]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:14:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:52 smithi172 ceph-mon[32456]: from='osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:14:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:52 smithi172 ceph-mon[32456]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:14:52.750 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:52 smithi173 ceph-mon[32906]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:14:52.750 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:52 smithi173 ceph-mon[32906]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:14:52.750 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:52.750 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:52 smithi173 ceph-mon[32906]: from='osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:14:52.750 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:52 smithi173 ceph-mon[32906]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:14:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:52 smithi037 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:14:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:52 smithi037 ceph-mon[29404]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:14:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:52 smithi037 ceph-mon[29404]: from='osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:14:52.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:52 smithi037 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:14:53.340 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:14:53.340 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:14:53.340 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T18:14:53.340 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T18:14:53.340 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113172 s, 927 MB/s 2024-04-23T18:14:53.340 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T18:14:53.661 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:53.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:14:53.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:14:53.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: from='osd.4 ' entity='osd.4' 2024-04-23T18:14:53.662 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:53 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:14:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:14:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: from='osd.4 ' entity='osd.4' 2024-04-23T18:14:53.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.839 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4[43831]: 2024-04-23T18:14:53.403+0000 7fe8227b6700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T18:14:53.839 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:14:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4[43831]: 2024-04-23T18:14:53.525+0000 7fe81a5a1700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: from='osd.4 ' entity='osd.4' 2024-04-23T18:14:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:53 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:54.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-04-23T18:14:54.274 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:14:54.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:54.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:54.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:54 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:54.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:54.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:54 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:54.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:14:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:14:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:54.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:14:54.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:54.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:54.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:54.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:54 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi172 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi172 to 8544M 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] boot 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:55.620 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:55.621 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:55.621 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:55.621 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi172 2024-04-23T18:14:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi172 to 8544M 2024-04-23T18:14:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] boot 2024-04-23T18:14:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:14:55.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:55.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:55.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:55.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:55.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 -- 172.21.15.173:0/1410315553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc1087b0 msgr2=0x7f15bc10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 --2- 172.21.15.173:0/1410315553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc1087b0 0x7f15bc10ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15b400b0a0 tx=0x7f15b400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 -- 172.21.15.173:0/1410315553 shutdown_connections 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 --2- 172.21.15.173:0/1410315553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc103550 0x7f15bc105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 --2- 172.21.15.173:0/1410315553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc1087b0 0x7f15bc10ac20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 --2- 172.21.15.173:0/1410315553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc105e80 0x7f15bc108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:55.841 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 -- 172.21.15.173:0/1410315553 >> 172.21.15.173:0/1410315553 conn(0x7f15bc0fcf60 msgr2=0x7f15bc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:14:55.842 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 -- 172.21.15.173:0/1410315553 shutdown_connections 2024-04-23T18:14:55.842 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 -- 172.21.15.173:0/1410315553 wait complete. 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.847+0000 7f15c08cc700 1 Processor -- start 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 -- start start 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 0x7f15bc102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc105e80 0x7f15bc101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc1087b0 0x7f15bc101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10e550 con 0x7f15bc105e80 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10e3d0 con 0x7f15bc1087b0 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15c08cc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10e6d0 con 0x7f15bc103550 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 0x7f15bc102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 0x7f15bc102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:32874/0 (socket says 172.21.15.173:32874) 2024-04-23T18:14:55.844 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 -- 172.21.15.173:0/569465935 learned_addr learned my addr 172.21.15.173:0/569465935 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15bad9d700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc1087b0 0x7f15bc101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15b1d9b700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc105e80 0x7f15bc101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 -- 172.21.15.173:0/569465935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc1087b0 msgr2=0x7f15bc101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc1087b0 0x7f15bc101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 -- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc105e80 msgr2=0x7f15bc101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc105e80 0x7f15bc101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15ba59c700 1 -- 172.21.15.173:0/569465935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc1022f0 con 0x7f15bc103550 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15b1d9b700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc105e80 0x7f15bc101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.848+0000 7f15bad9d700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc1087b0 0x7f15bc101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.849+0000 7f15ba59c700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 0x7f15bc102eb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00d950 tx=0x7f15ac00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.849+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac016070 con 0x7f15bc103550 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.849+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc1a2d20 con 0x7f15bc103550 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.849+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac010460 con 0x7f15bc103550 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.849+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00f620 con 0x7f15bc103550 2024-04-23T18:14:55.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.849+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a31b0 con 0x7f15bc103550 2024-04-23T18:14:55.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.850+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc02df80 con 0x7f15bc103550 2024-04-23T18:14:55.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.851+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f15ac019070 con 0x7f15bc103550 2024-04-23T18:14:55.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.851+0000 7f15b3fff700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f15a80a3c40 0x7f15a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:14:55.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.851+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f15ac0cefe0 con 0x7f15bc103550 2024-04-23T18:14:55.847 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.852+0000 7f15b1d9b700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f15a80a3c40 0x7f15a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:14:55.847 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.852+0000 7f15b1d9b700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f15a80a3c40 0x7f15a80a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15a400bcf0 tx=0x7f15a400b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:14:55.850 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:55.856+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15ac09ca30 con 0x7f15bc103550 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi172 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi172 to 8544M 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: osd.4 [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] boot 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:14:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:14:56.052 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:14:56.059+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f15bc062e80 con 0x7f15a80a3c40 2024-04-23T18:14:56.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:56 smithi173 ceph-mon[32906]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:14:56.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:56 smithi173 ceph-mon[32906]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:14:56.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:56 smithi173 ceph-mon[32906]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:56.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:56.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:56.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:56 smithi172 ceph-mon[32456]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:14:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:56 smithi172 ceph-mon[32456]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:14:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:56 smithi172 ceph-mon[32456]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:57.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:56 smithi037 ceph-mon[29404]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:14:57.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:56 smithi037 ceph-mon[29404]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:14:57.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:56 smithi037 ceph-mon[29404]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:14:57.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:14:57.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:14:57.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:14:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:58 smithi037 ceph-mon[29404]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:14:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/4176838379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]: dispatch 2024-04-23T18:14:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:58 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]: dispatch 2024-04-23T18:14:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:58 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]': finished 2024-04-23T18:14:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:58 smithi037 ceph-mon[29404]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:14:59.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:14:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:58 smithi172 ceph-mon[32456]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:14:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:58 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/4176838379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]: dispatch 2024-04-23T18:14:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:58 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]: dispatch 2024-04-23T18:14:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:58 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]': finished 2024-04-23T18:14:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:58 smithi172 ceph-mon[32456]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:14:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:14:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:58 smithi173 ceph-mon[32906]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:14:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:58 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/4176838379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]: dispatch 2024-04-23T18:14:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:58 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]: dispatch 2024-04-23T18:14:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:58 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0152f97-2416-42e9-b0a0-6a7389cb0c02"}]': finished 2024-04-23T18:14:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:58 smithi173 ceph-mon[32906]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:14:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:00.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:14:59 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/4260530948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:00.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:14:59 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/4260530948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:14:59 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/4260530948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:01.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:00 smithi037 ceph-mon[29404]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:01.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:00 smithi172 ceph-mon[32456]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:01.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:00 smithi173 ceph-mon[32906]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:03.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:02 smithi172 ceph-mon[32456]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:03.169 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:02 smithi173 ceph-mon[32906]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:03.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:02 smithi037 ceph-mon[29404]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:05.003 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:04 smithi173 ceph-mon[32906]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:05.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:04 smithi172 ceph-mon[32456]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:05.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:04 smithi037 ceph-mon[29404]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:06.938 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:06 smithi173 ceph-mon[32906]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:06.938 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:06 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:15:06.938 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:06 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:06.938 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:06 smithi173 ceph-mon[32906]: Deploying daemon osd.5 on smithi173 2024-04-23T18:15:07.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:06 smithi172 ceph-mon[32456]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:07.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:06 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:15:07.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:06 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:07.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:06 smithi172 ceph-mon[32456]: Deploying daemon osd.5 on smithi173 2024-04-23T18:15:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:06 smithi037 ceph-mon[29404]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:06 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:15:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:06 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:07.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:06 smithi037 ceph-mon[29404]: Deploying daemon osd.5 on smithi173 2024-04-23T18:15:09.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:08 smithi173 ceph-mon[32906]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:09.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:08 smithi172 ceph-mon[32456]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:09.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:08 smithi037 ceph-mon[29404]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:10.967 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:10 smithi173 ceph-mon[32906]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:11.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:10 smithi172 ceph-mon[32456]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:11.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:10 smithi037 ceph-mon[29404]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:12.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:12 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:12.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:12 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:12.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:12 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:12.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:12 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:12.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:12 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:12.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:12 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:12.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:12 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:12.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:12 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:12.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:12 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:13.794 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:13 smithi173 ceph-mon[32906]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:13 smithi172 ceph-mon[32456]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:13 smithi037 ceph-mon[29404]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:14.972 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.977+0000 7f15b3fff700 1 -- 172.21.15.173:0/569465935 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f15bc062e80 con 0x7f15a80a3c40 2024-04-23T18:15:14.973 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 5 on host 'smithi173' 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f15a80a3c40 msgr2=0x7f15a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f15a80a3c40 0x7f15a80a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15a400bcf0 tx=0x7f15a400b2c0 comp rx=0 tx=0).stop 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 msgr2=0x7f15bc102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 0x7f15bc102eb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00d950 tx=0x7f15ac00dd10 comp rx=0 tx=0).stop 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 shutdown_connections 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f15a80a3c40 0x7f15a80a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15bc1087b0 0x7f15bc101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15bc103550 0x7f15bc102eb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 --2- 172.21.15.173:0/569465935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15bc105e80 0x7f15bc101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:14.974 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 >> 172.21.15.173:0/569465935 conn(0x7f15bc0fcf60 msgr2=0x7f15bc103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:14.975 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 shutdown_connections 2024-04-23T18:15:14.975 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:15:14.980+0000 7f15c08cc700 1 -- 172.21.15.173:0/569465935 wait complete. 2024-04-23T18:15:15.710 DEBUG:teuthology.orchestra.run.smithi173:osd.5> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T18:15:15.713 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-04-23T18:15:15.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:15:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:15 smithi172 ceph-mon[32456]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:15 smithi172 ceph-mon[32456]: from='osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:15:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:15 smithi172 ceph-mon[32456]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:15:15.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:15 smithi173 ceph-mon[32906]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:15.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:15.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:15.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:15 smithi173 ceph-mon[32906]: from='osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:15:15.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:15 smithi173 ceph-mon[32906]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:15:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:15 smithi037 ceph-mon[29404]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:15 smithi037 ceph-mon[29404]: from='osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:15:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:15 smithi037 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:15:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:15:17.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: from='osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:17.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:16 smithi173 ceph-mon[32906]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: from='osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:17.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:16 smithi037 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:15:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:15:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: from='osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:15:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:15:17.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:17.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:16 smithi172 ceph-mon[32456]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:15:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:15:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:15:17.958 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-23T18:15:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-23T18:15:17.958 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111165 s, 943 MB/s 2024-04-23T18:15:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-23T18:15:17.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:17 smithi173 ceph-mon[32906]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:15:17.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:17 smithi173 ceph-mon[32906]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:15:17.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:17 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:17.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:17 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:17.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:17 smithi173 ceph-mon[32906]: from='osd.5 ' entity='osd.5' 2024-04-23T18:15:18.000 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:15:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:15:17.872+0000 7fd9c28df700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T18:15:18.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:17 smithi037 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:15:18.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:17 smithi037 ceph-mon[29404]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:15:18.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:18.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:17 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:18.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:17 smithi037 ceph-mon[29404]: from='osd.5 ' entity='osd.5' 2024-04-23T18:15:18.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:17 smithi172 ceph-mon[32456]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:15:18.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:17 smithi172 ceph-mon[32456]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:15:18.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:17 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:18.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:17 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:18.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:17 smithi172 ceph-mon[32456]: from='osd.5 ' entity='osd.5' 2024-04-23T18:15:18.355 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:15:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:15:18.005+0000 7fd9ba6ca700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:15:18.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-04-23T18:15:18.941 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:15:19.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:18 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:15:19.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:18 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:15:19.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:18 smithi173 ceph-mon[32906]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:19.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:18 smithi173 ceph-mon[32906]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T18:15:19.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:19.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:18 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:15:19.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:18 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:15:19.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:18 smithi037 ceph-mon[29404]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:19.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:18 smithi037 ceph-mon[29404]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T18:15:19.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:19.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:18 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:15:19.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:18 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:15:19.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:18 smithi172 ceph-mon[32456]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:19.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:18 smithi172 ceph-mon[32456]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T18:15:19.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] boot 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi173 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-23T18:15:20.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:19 smithi037 ceph-mon[29404]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] boot 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi173 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:20.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:19 smithi172 ceph-mon[32456]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:15:20.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: osd.5 [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] boot 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi173 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.411 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:20.412 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:19 smithi173 ceph-mon[32906]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:15:20.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 -- 172.21.15.37:0/888960847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc108790 msgr2=0x7f0bdc10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:20.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 --2- 172.21.15.37:0/888960847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc108790 0x7f0bdc10ac00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400b0a0 tx=0x7f0bd400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:15:20.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 -- 172.21.15.37:0/888960847 shutdown_connections 2024-04-23T18:15:20.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 --2- 172.21.15.37:0/888960847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc108790 0x7f0bdc10ac00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 --2- 172.21.15.37:0/888960847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc103530 0x7f0bdc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 --2- 172.21.15.37:0/888960847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc105e60 0x7f0bdc108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 -- 172.21.15.37:0/888960847 >> 172.21.15.37:0/888960847 conn(0x7f0bdc0fcf40 msgr2=0x7f0bdc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 -- 172.21.15.37:0/888960847 shutdown_connections 2024-04-23T18:15:20.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.530+0000 7f0be2a40700 1 -- 172.21.15.37:0/888960847 wait complete. 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 Processor -- start 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 -- start start 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 0x7f0bdc102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc105e60 0x7f0bdc1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc108790 0x7f0bdc101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10e580 con 0x7f0bdc103530 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10e400 con 0x7f0bdc108790 2024-04-23T18:15:20.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0be2a40700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10e700 con 0x7f0bdc105e60 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0bdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 0x7f0bdc102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.531+0000 7f0bdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 0x7f0bdc102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38766/0 (socket says 172.21.15.37:38766) 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 -- 172.21.15.37:0/1513963085 learned_addr learned my addr 172.21.15.37:0/1513963085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdb7fe700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc105e60 0x7f0bdc1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0be0fdd700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc108790 0x7f0bdc101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 -- 172.21.15.37:0/1513963085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc105e60 msgr2=0x7f0bdc1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc105e60 0x7f0bdc1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 -- 172.21.15.37:0/1513963085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc108790 msgr2=0x7f0bdc101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc108790 0x7f0bdc101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 -- 172.21.15.37:0/1513963085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1022d0 con 0x7f0bdc103530 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0be0fdd700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc108790 0x7f0bdc101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bdbfff700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 0x7f0bdc102e90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b530 tx=0x7f0bcc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:20.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc015020 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1a2db0 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc011ce0 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc010e40 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.532+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1a3280 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.533+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc069730 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.534+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0bcc018020 con 0x7f0bdc103530 2024-04-23T18:15:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.535+0000 7f0bd97fa700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0bc40a3c40 0x7f0bc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.535+0000 7f0bdb7fe700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0bc40a3c40 0x7f0bc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.535+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f0bcc0cf310 con 0x7f0bdc103530 2024-04-23T18:15:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.535+0000 7f0bdb7fe700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0bc40a3c40 0x7f0bc40a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005950 tx=0x7f0bd000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.539+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bcc01f020 con 0x7f0bdc103530 2024-04-23T18:15:20.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:20.757+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0bdc062e80 con 0x7f0bc40a3c40 2024-04-23T18:15:21.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:21 smithi037 ceph-mon[29404]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:21.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:15:21.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:15:21.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:21.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:21 smithi037 ceph-mon[29404]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:15:21.782 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:21 smithi173 ceph-mon[32906]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:21.783 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:15:21.783 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:15:21.783 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:21.783 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:21 smithi173 ceph-mon[32906]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:15:21.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:21 smithi172 ceph-mon[32456]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:15:21.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:15:21.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:15:21.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:21.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:21 smithi172 ceph-mon[32456]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:15:22.560 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:22 smithi037 ceph-mon[29404]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:15:22.560 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:22 smithi037 ceph-mon[29404]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:15:22.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:22 smithi172 ceph-mon[32456]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:15:22.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:22 smithi172 ceph-mon[32456]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:15:22.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:22 smithi173 ceph-mon[32906]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:15:22.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:22 smithi173 ceph-mon[32906]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:15:23.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:23.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-23T18:15:23.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T18:15:23.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/309853553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bc899de-4661-4c4f-b990-07db5cf3bd6e"}]: dispatch 2024-04-23T18:15:23.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/309853553' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bc899de-4661-4c4f-b990-07db5cf3bd6e"}]': finished 2024-04-23T18:15:23.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:15:23.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/309853553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bc899de-4661-4c4f-b990-07db5cf3bd6e"}]: dispatch 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/309853553' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bc899de-4661-4c4f-b990-07db5cf3bd6e"}]': finished 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:15:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:23.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:23.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-23T18:15:23.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T18:15:23.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/309853553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bc899de-4661-4c4f-b990-07db5cf3bd6e"}]: dispatch 2024-04-23T18:15:23.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/309853553' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bc899de-4661-4c4f-b990-07db5cf3bd6e"}]': finished 2024-04-23T18:15:23.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:15:23.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:24.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:24 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1505805873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:24.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:24 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1505805873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:24.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:24 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1505805873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:25.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:25 smithi172 ceph-mon[32456]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:25.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:25 smithi173 ceph-mon[32906]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:25.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:25 smithi037 ceph-mon[29404]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:27.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:27 smithi037 ceph-mon[29404]: pgmap v147: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:27.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:27 smithi172 ceph-mon[32456]: pgmap v147: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:27.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:27 smithi173 ceph-mon[32906]: pgmap v147: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T18:15:28.828 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:28 smithi037 ceph-mon[29404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-23T18:15:28.828 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:28 smithi037 ceph-mon[29404]: Cluster is now healthy 2024-04-23T18:15:28.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:28 smithi172 ceph-mon[32456]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-23T18:15:28.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:28 smithi172 ceph-mon[32456]: Cluster is now healthy 2024-04-23T18:15:28.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:28 smithi173 ceph-mon[32906]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-23T18:15:28.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:28 smithi173 ceph-mon[32906]: Cluster is now healthy 2024-04-23T18:15:29.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:29 smithi037 ceph-mon[29404]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:15:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:29 smithi172 ceph-mon[32456]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:15:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:29 smithi173 ceph-mon[32906]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:15:31.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:31 smithi037 ceph-mon[29404]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-23T18:15:31.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:15:31.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:31.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:31 smithi172 ceph-mon[32456]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-23T18:15:31.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:15:31.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:31.851 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:31 smithi173 ceph-mon[32906]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-23T18:15:31.851 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:15:31.851 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:32.833 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:32 smithi037 ceph-mon[29404]: Deploying daemon osd.6 on smithi037 2024-04-23T18:15:32.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:32 smithi172 ceph-mon[32456]: Deploying daemon osd.6 on smithi037 2024-04-23T18:15:32.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:32 smithi173 ceph-mon[32906]: Deploying daemon osd.6 on smithi037 2024-04-23T18:15:33.818 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:33 smithi037 ceph-mon[29404]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-23T18:15:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:33 smithi172 ceph-mon[32456]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-23T18:15:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:33 smithi173 ceph-mon[32906]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-23T18:15:35.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:35 smithi037 ceph-mon[29404]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-23T18:15:35.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:35 smithi172 ceph-mon[32456]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-23T18:15:35.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:35 smithi173 ceph-mon[32906]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:37 smithi172 ceph-mon[32456]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:37 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:37 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:37 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:37.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:37 smithi173 ceph-mon[32906]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:15:37.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:37.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:37 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:37 smithi037 ceph-mon[29404]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:15:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:37.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:37 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:39.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:39 smithi172 ceph-mon[32456]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:15:39.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:39.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:39.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:39 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:39 smithi173 ceph-mon[32906]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:15:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:39 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:39 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:39 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:39 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:39.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:39 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:39 smithi037 ceph-mon[29404]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:15:39.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:39.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:39.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:39.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:39 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:40.358 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-04-23T18:15:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.363+0000 7f0bd97fa700 1 -- 172.21.15.37:0/1513963085 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0bdc062e80 con 0x7f0bc40a3c40 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0bc40a3c40 msgr2=0x7f0bc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0bc40a3c40 0x7f0bc40a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005950 tx=0x7f0bd000a300 comp rx=0 tx=0).stop 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 msgr2=0x7f0bdc102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 0x7f0bdc102e90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b530 tx=0x7f0bcc00b840 comp rx=0 tx=0).stop 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 shutdown_connections 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0bc40a3c40 0x7f0bc40a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc103530 0x7f0bdc102e90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bdc108790 0x7f0bdc101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 --2- 172.21.15.37:0/1513963085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc105e60 0x7f0bdc1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 >> 172.21.15.37:0/1513963085 conn(0x7f0bdc0fcf40 msgr2=0x7f0bdc103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.365+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 shutdown_connections 2024-04-23T18:15:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:15:40.366+0000 7f0be2a40700 1 -- 172.21.15.37:0/1513963085 wait complete. 2024-04-23T18:15:41.039 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.6.service 2024-04-23T18:15:41.041 INFO:tasks.cephadm:Deploying osd.7 on smithi172 with /dev/vg_nvme/lv_2... 2024-04-23T18:15:41.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:15:41.287 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:15:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:41 smithi172 ceph-mon[32456]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:41 smithi172 ceph-mon[32456]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:15:41.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:41 smithi173 ceph-mon[32906]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:41.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:41.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:41.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:41.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:41 smithi173 ceph-mon[32906]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:15:41.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:41 smithi037 ceph-mon[29404]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:41.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:41.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:41.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:15:41.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:41 smithi037 ceph-mon[29404]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:15:42.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:42 smithi037 ceph-mon[29404]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:15:42.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:42 smithi037 ceph-mon[29404]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:15:42.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:42 smithi037 ceph-mon[29404]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:15:42.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:42 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:42.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:42 smithi172 ceph-mon[32456]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:15:42.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:42 smithi172 ceph-mon[32456]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:15:42.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:42 smithi172 ceph-mon[32456]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:15:42.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:42 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:42.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:42 smithi173 ceph-mon[32906]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:15:42.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:42 smithi173 ceph-mon[32906]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:15:42.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:42 smithi173 ceph-mon[32906]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:15:42.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:42 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.234 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:15:43.234 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:15:43.234 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-23T18:15:43.234 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-23T18:15:43.235 INFO:teuthology.orchestra.run.smithi172.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131838 s, 795 MB/s 2024-04-23T18:15:43.235 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-23T18:15:43.363 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:43 smithi172 ceph-mon[32456]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:43.363 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:43 smithi172 ceph-mon[32456]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:15:43.363 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:43 smithi172 ceph-mon[32456]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T18:15:43.363 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.363 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:43 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.363 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:43 smithi172 ceph-mon[32456]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' 2024-04-23T18:15:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:43 smithi173 ceph-mon[32906]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:43 smithi173 ceph-mon[32906]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:15:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:43 smithi173 ceph-mon[32906]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T18:15:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:43 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:43 smithi173 ceph-mon[32906]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' 2024-04-23T18:15:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-mon[29404]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-mon[29404]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:15:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-mon[29404]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T18:15:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:43.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-mon[29404]: from='osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067]' entity='osd.6' 2024-04-23T18:15:43.621 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:15:43.237+0000 7f0a23377700 -1 osd.6 0 waiting for initial osdmap 2024-04-23T18:15:43.622 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:15:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:15:43.328+0000 7f0a1a961700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:15:43.856 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-04-23T18:15:44.098 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] boot 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:15:44.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:44.472 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] boot 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:15:44.473 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:44.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:15:44.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:15:44.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: osd.6 [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] boot 2024-04-23T18:15:44.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:15:44.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:15:45.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/1714597261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf41019a0 msgr2=0x7f3bf4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:45.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/1714597261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf41019a0 0x7f3bf4101dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3be40099c0 tx=0x7f3be4009cd0 comp rx=0 tx=0).stop 2024-04-23T18:15:45.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/1714597261 shutdown_connections 2024-04-23T18:15:45.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/1714597261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bf4103540 0x7f3bf4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:45.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/1714597261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf41019a0 0x7f3bf4101dc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/1714597261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf4102ba0 0x7f3bf4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/1714597261 >> 172.21.15.172:0/1714597261 conn(0x7f3bf40fcf40 msgr2=0x7f3bf40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/1714597261 shutdown_connections 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.538+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/1714597261 wait complete. 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 Processor -- start 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 -- start start 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf41019a0 0x7f3bf4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bf4102ba0 0x7f3bf4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 0x7f3bf41a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410d490 con 0x7f3bf41019a0 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410d310 con 0x7f3bf4103540 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bfb9c3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410d610 con 0x7f3bf4102ba0 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bf9f60700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 0x7f3bf41a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:45.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bf9f60700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 0x7f3bf41a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39980/0 (socket says 172.21.15.172:39980) 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bf9f60700 1 -- 172.21.15.172:0/593936561 learned_addr learned my addr 172.21.15.172:0/593936561 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.539+0000 7f3bf975f700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf41019a0 0x7f3bf4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf8f5e700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bf4102ba0 0x7f3bf4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf9f60700 1 -- 172.21.15.172:0/593936561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bf4102ba0 msgr2=0x7f3bf4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf9f60700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bf4102ba0 0x7f3bf4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf9f60700 1 -- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf41019a0 msgr2=0x7f3bf4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf9f60700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf41019a0 0x7f3bf4198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf9f60700 1 -- 172.21.15.172:0/593936561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf41a0910 con 0x7f3bf4103540 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf975f700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf41019a0 0x7f3bf4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bf9f60700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 0x7f3bf41a0240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00b970 tx=0x7f3bec00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec019070 con 0x7f3bf4103540 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf41a0c00 con 0x7f3bf4103540 2024-04-23T18:15:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bec014440 con 0x7f3bf4103540 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00ce60 con 0x7f3bf4103540 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.540+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41a1140 con 0x7f3bf4103540 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.541+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf4069730 con 0x7f3bf4103540 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.543+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3bec01c070 con 0x7f3bf4103540 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.543+0000 7f3bea7fc700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3be00a3cc0 0x7f3be00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.543+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f3bec0cf080 con 0x7f3bf4103540 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.544+0000 7f3bf975f700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3be00a3cc0 0x7f3be00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.544+0000 7f3bf975f700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3be00a3cc0 0x7f3be00a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009990 tx=0x7f3be4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:15:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.547+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bec09c6d0 con 0x7f3bf4103540 2024-04-23T18:15:45.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:45 smithi172 ceph-mon[32456]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:45.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:45 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi037 2024-04-23T18:15:45.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:45 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-23T18:15:45.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:45 smithi037 ceph-mon[29404]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:45.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:45 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi037 2024-04-23T18:15:45.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:45 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-23T18:15:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:15:45.755+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3bf406ae30 con 0x7f3be00a3cc0 2024-04-23T18:15:45.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:45 smithi173 ceph-mon[32906]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:15:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:45 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi037 2024-04-23T18:15:45.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:45 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-23T18:15:46.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:46 smithi037 ceph-mon[29404]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T18:15:46.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:15:46.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:15:46.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:46.673 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:46 smithi172 ceph-mon[32456]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T18:15:46.674 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:15:46.674 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:15:46.674 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:46.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:46 smithi173 ceph-mon[32906]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T18:15:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:15:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:15:46.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:47.505 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:47 smithi172 ceph-mon[32456]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:15:47.505 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:47 smithi172 ceph-mon[32456]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:47.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:47 smithi037 ceph-mon[29404]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:15:47.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:47 smithi037 ceph-mon[29404]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:47.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:47 smithi173 ceph-mon[32906]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:15:47.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:47 smithi173 ceph-mon[32906]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:48.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:48 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/234478416' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]: dispatch 2024-04-23T18:15:48.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:48 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]: dispatch 2024-04-23T18:15:48.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:48 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]': finished 2024-04-23T18:15:48.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:48 smithi172 ceph-mon[32456]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:15:48.664 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:15:48.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:48 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/234478416' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]: dispatch 2024-04-23T18:15:48.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:48 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]: dispatch 2024-04-23T18:15:48.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:48 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]': finished 2024-04-23T18:15:48.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:48 smithi173 ceph-mon[32906]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:15:48.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:15:48.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:48 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/234478416' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]: dispatch 2024-04-23T18:15:48.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:48 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]: dispatch 2024-04-23T18:15:48.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:48 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb4b1b9-9fb4-475f-a40c-79ff6ba58324"}]': finished 2024-04-23T18:15:48.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:48 smithi037 ceph-mon[29404]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:15:48.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:15:49.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:49 smithi173 ceph-mon[32906]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:49.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:49 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/751379948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:49.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:49 smithi172 ceph-mon[32456]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:49.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:49 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/751379948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:49 smithi037 ceph-mon[29404]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:49 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/751379948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:15:51.640 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:51 smithi172 ceph-mon[32456]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:51.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:51 smithi173 ceph-mon[32906]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:51.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:51 smithi037 ceph-mon[29404]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:53.671 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:53 smithi172 ceph-mon[32456]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:53.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:53 smithi173 ceph-mon[32906]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:53.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:53 smithi037 ceph-mon[29404]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:55.648 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:55 smithi172 ceph-mon[32456]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:55.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:55 smithi173 ceph-mon[32906]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:55.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:55 smithi037 ceph-mon[29404]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:56.559 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:15:56.559 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:56.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:15:56.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:56.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:15:56.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:15:57.496 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:57 smithi172 ceph-mon[32456]: Deploying daemon osd.7 on smithi172 2024-04-23T18:15:57.497 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:57 smithi172 ceph-mon[32456]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:57.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:57 smithi173 ceph-mon[32906]: Deploying daemon osd.7 on smithi172 2024-04-23T18:15:57.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:57 smithi173 ceph-mon[32906]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:57.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:57 smithi037 ceph-mon[29404]: Deploying daemon osd.7 on smithi172 2024-04-23T18:15:57.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:57 smithi037 ceph-mon[29404]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:59.771 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:15:59 smithi172 ceph-mon[32456]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:59.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:15:59 smithi173 ceph-mon[32906]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:15:59.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:15:59 smithi037 ceph-mon[29404]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:01.531 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:01 smithi172 ceph-mon[32456]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:01.531 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:01.531 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:01.531 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:01 smithi037 ceph-mon[29404]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:01.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:01.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:01 smithi173 ceph-mon[32906]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:01.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:01.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:01.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:03.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:03 smithi172 ceph-mon[32456]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:03.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:03 smithi173 ceph-mon[32906]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:03.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:03 smithi037 ceph-mon[29404]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:04.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.575+0000 7f3bea7fc700 1 -- 172.21.15.172:0/593936561 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bf406ae30 con 0x7f3be00a3cc0 2024-04-23T18:16:04.569 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 7 on host 'smithi172' 2024-04-23T18:16:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3be00a3cc0 msgr2=0x7f3be00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3be00a3cc0 0x7f3be00a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009990 tx=0x7f3be4005fb0 comp rx=0 tx=0).stop 2024-04-23T18:16:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 msgr2=0x7f3bf41a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 0x7f3bf41a0240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00b970 tx=0x7f3bec00bd30 comp rx=0 tx=0).stop 2024-04-23T18:16:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 shutdown_connections 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3be00a3cc0 0x7f3be00a6180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bf41019a0 0x7f3bf4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bf4103540 0x7f3bf41a0240 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 --2- 172.21.15.172:0/593936561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bf4102ba0 0x7f3bf4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 >> 172.21.15.172:0/593936561 conn(0x7f3bf40fcf40 msgr2=0x7f3bf406a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 shutdown_connections 2024-04-23T18:16:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:16:04.577+0000 7f3bfb9c3700 1 -- 172.21.15.172:0/593936561 wait complete. 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:05.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:16:05.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:04 smithi172 ceph-mon[32456]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:16:05.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:04 smithi173 ceph-mon[32906]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:16:05.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:04 smithi037 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:16:05.168 DEBUG:teuthology.orchestra.run.smithi172:osd.7> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.7.service 2024-04-23T18:16:05.170 INFO:tasks.cephadm:Deploying osd.8 on smithi173 with /dev/vg_nvme/lv_2... 2024-04-23T18:16:05.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:16:05.415 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:16:06.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:06 smithi172 ceph-mon[32456]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:16:06.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:06 smithi172 ceph-mon[32456]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:16:06.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:06 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:06.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:06 smithi172 ceph-mon[32456]: from='osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:16:06.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:06 smithi172 ceph-mon[32456]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:16:06.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:06 smithi173 ceph-mon[32906]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:16:06.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:06 smithi173 ceph-mon[32906]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:16:06.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:06 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:06.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:06 smithi173 ceph-mon[32906]: from='osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:16:06.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:06 smithi173 ceph-mon[32906]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:16:06.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:06 smithi037 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:16:06.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:06 smithi037 ceph-mon[29404]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:16:06.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:06 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:06.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:06 smithi037 ceph-mon[29404]: from='osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:16:06.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:06 smithi037 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:16:07.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-mon[32456]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:07.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-mon[32456]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:16:07.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-mon[32456]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:16:07.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:07.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:07.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-mon[32456]: from='osd.7 ' entity='osd.7' 2024-04-23T18:16:07.588 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:16:07.463+0000 7ff53729a700 -1 osd.7 0 waiting for initial osdmap 2024-04-23T18:16:07.637 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:16:07.637 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:16:07.637 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T18:16:07.637 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T18:16:07.637 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124613 s, 841 MB/s 2024-04-23T18:16:07.638 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T18:16:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:07 smithi173 ceph-mon[32906]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:07 smithi173 ceph-mon[32906]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:16:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:07 smithi173 ceph-mon[32906]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:16:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:07 smithi173 ceph-mon[32906]: from='osd.7 ' entity='osd.7' 2024-04-23T18:16:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:07 smithi037 ceph-mon[29404]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:07 smithi037 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:16:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:07 smithi037 ceph-mon[29404]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T18:16:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:07 smithi037 ceph-mon[29404]: from='osd.7 ' entity='osd.7' 2024-04-23T18:16:07.973 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:16:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:16:07.613+0000 7ff52e884700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:16:08.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-04-23T18:16:08.445 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: osdmap e52: 8 total, 7 up, 8 in 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:16:08.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: osdmap e52: 8 total, 7 up, 8 in 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:16:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:16:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: osdmap e52: 8 total, 7 up, 8 in 2024-04-23T18:16:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:08.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:09.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:09 smithi172 ceph-mon[32456]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:09.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:09 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi172 2024-04-23T18:16:09.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:09 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi172 to 5696M 2024-04-23T18:16:09.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:09 smithi172 ceph-mon[32456]: osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] boot 2024-04-23T18:16:09.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:09 smithi172 ceph-mon[32456]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:16:09.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:09 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:09.841 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:09 smithi173 ceph-mon[32906]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:09.841 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:09 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi172 2024-04-23T18:16:09.841 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:09 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi172 to 5696M 2024-04-23T18:16:09.841 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:09 smithi173 ceph-mon[32906]: osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] boot 2024-04-23T18:16:09.841 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:09 smithi173 ceph-mon[32906]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:16:09.841 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:09 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:09 smithi037 ceph-mon[29404]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:16:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:09 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi172 2024-04-23T18:16:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:09 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi172 to 5696M 2024-04-23T18:16:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:09 smithi037 ceph-mon[29404]: osd.7 [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] boot 2024-04-23T18:16:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:09 smithi037 ceph-mon[29404]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:16:09.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:09 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:16:09.966 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 -- 172.21.15.173:0/2034804598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101990 msgr2=0x7ff8e0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:09.966 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/2034804598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101990 0x7ff8e0101db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009990 tx=0x7ff8d0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.966 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 -- 172.21.15.173:0/2034804598 shutdown_connections 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/2034804598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8e0103530 0x7ff8e0109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/2034804598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0101990 0x7ff8e0101db0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/2034804598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e0102b90 0x7ff8e0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 -- 172.21.15.173:0/2034804598 >> 172.21.15.173:0/2034804598 conn(0x7ff8e00fcf30 msgr2=0x7ff8e00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 -- 172.21.15.173:0/2034804598 shutdown_connections 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.951+0000 7ff8e49d9700 1 -- 172.21.15.173:0/2034804598 wait complete. 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 Processor -- start 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 -- start start 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8e0101990 0x7ff8e0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e0102b90 0x7ff8e01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 0x7ff8e01a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010d420 con 0x7ff8e0101990 2024-04-23T18:16:09.967 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010d2a0 con 0x7ff8e0102b90 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8e49d9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010d5a0 con 0x7ff8e0103530 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 0x7ff8e01a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 0x7ff8e01a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:44462/0 (socket says 172.21.15.173:44462) 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 -- 172.21.15.173:0/1211323639 learned_addr learned my addr 172.21.15.173:0/1211323639 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ddd9b700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e0102b90 0x7ff8e01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8de59c700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8e0101990 0x7ff8e0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 -- 172.21.15.173:0/1211323639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e0102b90 msgr2=0x7ff8e01991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e0102b90 0x7ff8e01991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 -- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8e0101990 msgr2=0x7ff8e0198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8e0101990 0x7ff8e0198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 -- 172.21.15.173:0/1211323639 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e01a0890 con 0x7ff8e0103530 2024-04-23T18:16:09.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8ded9d700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 0x7ff8e01a01c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8007ee0 tx=0x7ff8d800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.952+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800ddd0 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.953+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d8004020 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.953+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e01a0b80 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.953+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8015660 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.953+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01a1070 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.953+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e0069730 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.955+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff8d80041d0 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.955+0000 7ff8cf7fe700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff8c80a3c40 0x7ff8c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.956+0000 7ff8de59c700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff8c80a3c40 0x7ff8c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.956+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7ff8d80cfbe0 con 0x7ff8e0103530 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.956+0000 7ff8de59c700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff8c80a3c40 0x7ff8c80a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b720 tx=0x7ff8d001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:09.969 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:09.959+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8d8010070 con 0x7ff8e0103530 2024-04-23T18:16:10.161 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:10.167+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff8e006b8e0 con 0x7ff8c80a3c40 2024-04-23T18:16:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:10 smithi172 ceph-mon[32456]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:16:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:16:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:16:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:10 smithi172 ceph-mon[32456]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:16:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:10 smithi173 ceph-mon[32906]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:16:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:16:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:16:10.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:10.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:10 smithi173 ceph-mon[32906]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:16:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:10 smithi037 ceph-mon[29404]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:16:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:16:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:16:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:10 smithi037 ceph-mon[29404]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:16:11.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:11 smithi173 ceph-mon[32906]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:11.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:11 smithi173 ceph-mon[32906]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:11.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:11 smithi173 ceph-mon[32906]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:16:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:11 smithi172 ceph-mon[32456]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:11 smithi172 ceph-mon[32456]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:11 smithi172 ceph-mon[32456]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:16:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:11 smithi037 ceph-mon[29404]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:11 smithi037 ceph-mon[29404]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:11 smithi037 ceph-mon[29404]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T18:16:13.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:12 smithi172 ceph-mon[32456]: osdmap e57: 8 total, 8 up, 8 in 2024-04-23T18:16:13.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:12 smithi172 ceph-mon[32456]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:13.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:12 smithi173 ceph-mon[32906]: osdmap e57: 8 total, 8 up, 8 in 2024-04-23T18:16:13.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:12 smithi173 ceph-mon[32906]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:13.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:12 smithi037 ceph-mon[29404]: osdmap e57: 8 total, 8 up, 8 in 2024-04-23T18:16:13.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:12 smithi037 ceph-mon[29404]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:14.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:13 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/1929791480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]: dispatch 2024-04-23T18:16:14.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:13 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]: dispatch 2024-04-23T18:16:14.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:13 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]': finished 2024-04-23T18:16:14.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:13 smithi037 ceph-mon[29404]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:16:14.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:14.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:13 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/4242722012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:16:14.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:13 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/1929791480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]: dispatch 2024-04-23T18:16:14.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:13 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]: dispatch 2024-04-23T18:16:14.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:13 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]': finished 2024-04-23T18:16:14.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:13 smithi172 ceph-mon[32456]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:16:14.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:14.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:13 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/4242722012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:16:14.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:13 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/1929791480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]: dispatch 2024-04-23T18:16:14.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:13 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]: dispatch 2024-04-23T18:16:14.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:13 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec2a2669-fc68-4265-b68b-be4f008e4c2c"}]': finished 2024-04-23T18:16:14.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:13 smithi173 ceph-mon[32906]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:16:14.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:14.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:13 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/4242722012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:16:15.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:14 smithi172 ceph-mon[32456]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:15.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:14 smithi173 ceph-mon[32906]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:15.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:14 smithi037 ceph-mon[29404]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:17.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:16 smithi173 ceph-mon[32906]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:17.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:16 smithi172 ceph-mon[32456]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:17.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:16 smithi037 ceph-mon[29404]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:19.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:18 smithi173 ceph-mon[32906]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:19.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:18 smithi172 ceph-mon[32456]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:19.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:18 smithi037 ceph-mon[29404]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:20 smithi173 ceph-mon[32906]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:20 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:20 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:21.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:20 smithi173 ceph-mon[32906]: Deploying daemon osd.8 on smithi173 2024-04-23T18:16:21.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:20 smithi172 ceph-mon[32456]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:21.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:20 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:16:21.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:20 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:21.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:20 smithi172 ceph-mon[32456]: Deploying daemon osd.8 on smithi173 2024-04-23T18:16:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:20 smithi037 ceph-mon[29404]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:20 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:16:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:20 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:21.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:20 smithi037 ceph-mon[29404]: Deploying daemon osd.8 on smithi173 2024-04-23T18:16:22.971 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:22 smithi173 ceph-mon[32906]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:23.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:22 smithi172 ceph-mon[32456]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:23.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:22 smithi037 ceph-mon[29404]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:25.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:24 smithi173 ceph-mon[32906]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:25.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:24 smithi172 ceph-mon[32456]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:25.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:24 smithi037 ceph-mon[29404]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:26.990 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:26.990 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:26.990 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:27.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:27.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:27.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:27.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:27.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:27.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:27.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:27 smithi173 ceph-mon[32906]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:28.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:27 smithi172 ceph-mon[32456]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:28.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:27 smithi037 ceph-mon[29404]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:29.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:28 smithi173 ceph-mon[32906]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:29.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:28 smithi172 ceph-mon[32456]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:29.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:28 smithi037 ceph-mon[29404]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:29.128 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.134+0000 7ff8cf7fe700 1 -- 172.21.15.173:0/1211323639 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8e006b8e0 con 0x7ff8c80a3c40 2024-04-23T18:16:29.128 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 8 on host 'smithi173' 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.136+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff8c80a3c40 msgr2=0x7ff8c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.136+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff8c80a3c40 0x7ff8c80a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b720 tx=0x7ff8d001a040 comp rx=0 tx=0).stop 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.136+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 msgr2=0x7ff8e01a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.136+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 0x7ff8e01a01c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8007ee0 tx=0x7ff8d800d3b0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 shutdown_connections 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff8c80a3c40 0x7ff8c80a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8e0101990 0x7ff8e0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8e0103530 0x7ff8e01a01c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 --2- 172.21.15.173:0/1211323639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8e0102b90 0x7ff8e01991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 >> 172.21.15.173:0/1211323639 conn(0x7ff8e00fcf30 msgr2=0x7ff8e006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 shutdown_connections 2024-04-23T18:16:29.131 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:16:29.137+0000 7ff8e49d9700 1 -- 172.21.15.173:0/1211323639 wait complete. 2024-04-23T18:16:29.749 DEBUG:teuthology.orchestra.run.smithi173:osd.8> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.8.service 2024-04-23T18:16:29.751 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-04-23T18:16:29.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:16:30.012 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:16:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:30 smithi037 ceph-mon[29404]: from='osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:16:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:30 smithi037 ceph-mon[29404]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:16:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.514 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.514 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.514 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:30 smithi173 ceph-mon[32906]: from='osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:16:30.514 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:30 smithi173 ceph-mon[32906]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:16:30.514 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.514 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:30 smithi172 ceph-mon[32456]: from='osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:16:30.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:30 smithi172 ceph-mon[32456]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:16:30.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:30.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:31.404 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:31 smithi037 ceph-mon[29404]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:31.405 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:31 smithi037 ceph-mon[29404]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:16:31.405 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:31 smithi037 ceph-mon[29404]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:16:31.405 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:31 smithi037 ceph-mon[29404]: from='osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:16:31.405 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:31.405 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:31 smithi037 ceph-mon[29404]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:16:31.419 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-mon[32906]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:31.419 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-mon[32906]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:16:31.419 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-mon[32906]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:16:31.420 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-mon[32906]: from='osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:16:31.420 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:31.420 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-mon[32906]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:16:31.420 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:16:31.137+0000 7feb17067700 -1 osd.8 0 waiting for initial osdmap 2024-04-23T18:16:31.420 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:16:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:16:31.148+0000 7feb12e5a700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:16:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:31 smithi172 ceph-mon[32456]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:31 smithi172 ceph-mon[32456]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:16:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:31 smithi172 ceph-mon[32456]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T18:16:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:31 smithi172 ceph-mon[32456]: from='osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:16:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:31.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:31 smithi172 ceph-mon[32456]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:16:32.119 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:16:32.119 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:16:32.119 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-23T18:16:32.119 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-23T18:16:32.119 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127223 s, 824 MB/s 2024-04-23T18:16:32.119 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-23T18:16:32.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:32 smithi037 ceph-mon[29404]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:16:32.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:32 smithi037 ceph-mon[29404]: osdmap e60: 9 total, 8 up, 9 in 2024-04-23T18:16:32.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:32 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:32.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:32 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:32.405 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:32 smithi172 ceph-mon[32456]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:16:32.405 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:32 smithi172 ceph-mon[32456]: osdmap e60: 9 total, 8 up, 9 in 2024-04-23T18:16:32.405 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:32 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:32.405 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:32 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:32 smithi173 ceph-mon[32906]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:16:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:32 smithi173 ceph-mon[32906]: osdmap e60: 9 total, 8 up, 9 in 2024-04-23T18:16:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:32 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:32 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:32.900 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-04-23T18:16:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:16:33.239 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:16:33.240 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:16:33.240 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:33.240 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:33.240 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] boot 2024-04-23T18:16:33.240 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:16:33.240 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:33.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:16:33.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:16:33.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:33.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:33.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] boot 2024-04-23T18:16:33.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:16:33.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: osd.8 [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] boot 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:16:33.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi173 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:34.402 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:34 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi173 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-23T18:16:34.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:34.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:34 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi173 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:34.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:34.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:34 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:34.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 -- 172.21.15.37:0/261813493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118104640 msgr2=0x7f411810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:34.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 --2- 172.21.15.37:0/261813493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118104640 0x7f411810aef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f411000b0d0 tx=0x7f411000b3e0 comp rx=0 tx=0).stop 2024-04-23T18:16:34.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 -- 172.21.15.37:0/261813493 shutdown_connections 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 --2- 172.21.15.37:0/261813493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118104640 0x7f411810aef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 --2- 172.21.15.37:0/261813493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4118102aa0 0x7f4118102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 --2- 172.21.15.37:0/261813493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4118103ca0 0x7f4118104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 -- 172.21.15.37:0/261813493 >> 172.21.15.37:0/261813493 conn(0x7f41180fe040 msgr2=0x7f4118100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 -- 172.21.15.37:0/261813493 shutdown_connections 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.882+0000 7f411fb7e700 1 -- 172.21.15.37:0/261813493 wait complete. 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 Processor -- start 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 -- start start 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4118102aa0 0x7f41180767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 0x7f4118074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4118104640 0x7f4118075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f411810e520 con 0x7f4118103ca0 2024-04-23T18:16:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f411810e3a0 con 0x7f4118104640 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411fb7e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f411810e6a0 con 0x7f4118102aa0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 0x7f4118074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 0x7f4118074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47420/0 (socket says 172.21.15.37:47420) 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 -- 172.21.15.37:0/3973001661 learned_addr learned my addr 172.21.15.37:0/3973001661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411e11b700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4118104640 0x7f4118075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d91a700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4118102aa0 0x7f41180767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 -- 172.21.15.37:0/3973001661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4118102aa0 msgr2=0x7f41180767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4118102aa0 0x7f41180767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 -- 172.21.15.37:0/3973001661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4118104640 msgr2=0x7f4118075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4118104640 0x7f4118075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 -- 172.21.15.37:0/3973001661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4118075c20 con 0x7f4118103ca0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.883+0000 7f411d119700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 0x7f4118074e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f410800d950 tx=0x7f410800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.884+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108016070 con 0x7f4118103ca0 2024-04-23T18:16:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.884+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108009df0 con 0x7f4118103ca0 2024-04-23T18:16:34.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.884+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181a2cc0 con 0x7f4118103ca0 2024-04-23T18:16:34.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.884+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410800f3c0 con 0x7f4118103ca0 2024-04-23T18:16:34.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.884+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181a3180 con 0x7f4118103ca0 2024-04-23T18:16:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.885+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4118069730 con 0x7f4118103ca0 2024-04-23T18:16:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.886+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4108019070 con 0x7f4118103ca0 2024-04-23T18:16:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.887+0000 7f410effd700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f41040a3c40 0x7f41040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.887+0000 7f411d91a700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f41040a3c40 0x7f41040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.887+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5858+0+0 (secure 0 0 0) 0x7f41080cf6c0 con 0x7f4118103ca0 2024-04-23T18:16:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.887+0000 7f411d91a700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f41040a3c40 0x7f41040a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4114005950 tx=0x7f41140058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:34.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:34.891+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f410809c990 con 0x7f4118103ca0 2024-04-23T18:16:35.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:35.107+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4118062e80 con 0x7f41040a3c40 2024-04-23T18:16:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:35 smithi037 ceph-mon[29404]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:35 smithi037 ceph-mon[29404]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:16:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:16:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:16:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:35.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:35 smithi172 ceph-mon[32456]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:35.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:35 smithi172 ceph-mon[32456]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:16:35.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:16:35.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:16:35.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:35.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:35 smithi173 ceph-mon[32906]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:35.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:35 smithi173 ceph-mon[32906]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:16:35.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:16:35.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:16:35.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:36 smithi037 ceph-mon[29404]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:36 smithi037 ceph-mon[29404]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:16:36.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:36 smithi172 ceph-mon[32456]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:36.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:36 smithi172 ceph-mon[32456]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:16:36.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:36 smithi173 ceph-mon[32906]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:16:36.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:36 smithi173 ceph-mon[32906]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T18:16:37.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:37 smithi172 ceph-mon[32456]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:16:37.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:37 smithi172 ceph-mon[32456]: osdmap e65: 9 total, 9 up, 9 in 2024-04-23T18:16:37.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:37 smithi173 ceph-mon[32906]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:16:37.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:37 smithi173 ceph-mon[32906]: osdmap e65: 9 total, 9 up, 9 in 2024-04-23T18:16:37.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:37 smithi037 ceph-mon[29404]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:16:37.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:37 smithi037 ceph-mon[29404]: osdmap e65: 9 total, 9 up, 9 in 2024-04-23T18:16:38.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:38 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/733420699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7eddb6e9-0b34-4a18-961f-0134670a2581"}]: dispatch 2024-04-23T18:16:38.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:38 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/733420699' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7eddb6e9-0b34-4a18-961f-0134670a2581"}]': finished 2024-04-23T18:16:38.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:38 smithi172 ceph-mon[32456]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:16:38.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:38 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:38.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/733420699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7eddb6e9-0b34-4a18-961f-0134670a2581"}]: dispatch 2024-04-23T18:16:38.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/733420699' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7eddb6e9-0b34-4a18-961f-0134670a2581"}]': finished 2024-04-23T18:16:38.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:38 smithi173 ceph-mon[32906]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:16:38.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:38 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:38.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/733420699' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7eddb6e9-0b34-4a18-961f-0134670a2581"}]: dispatch 2024-04-23T18:16:38.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/733420699' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7eddb6e9-0b34-4a18-961f-0134670a2581"}]': finished 2024-04-23T18:16:38.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:38 smithi037 ceph-mon[29404]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:16:38.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:38 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:39.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:39 smithi172 ceph-mon[32456]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:16:39.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1427715803' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:16:39.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:39 smithi173 ceph-mon[32906]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:16:39.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1427715803' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:16:39.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:39 smithi037 ceph-mon[29404]: pgmap v205: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:16:39.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1427715803' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:16:40.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:16:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:16:39.802+0000 7f82b7d32700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-23T18:16:41.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:41 smithi172 ceph-mon[32456]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-23T18:16:41.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:41 smithi173 ceph-mon[32906]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-23T18:16:41.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:41 smithi037 ceph-mon[29404]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-23T18:16:43.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:43 smithi037 ceph-mon[29404]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:43.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:43 smithi172 ceph-mon[32456]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:43.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:43 smithi173 ceph-mon[32906]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:45.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:45 smithi037 ceph-mon[29404]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:45.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:45 smithi172 ceph-mon[32456]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:45.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:45 smithi173 ceph-mon[32906]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:46.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:16:46.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:46.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:46 smithi037 ceph-mon[29404]: Deploying daemon osd.9 on smithi037 2024-04-23T18:16:46.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:16:46.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:46.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:46 smithi172 ceph-mon[32456]: Deploying daemon osd.9 on smithi037 2024-04-23T18:16:46.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:16:46.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:46.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:46 smithi173 ceph-mon[32906]: Deploying daemon osd.9 on smithi037 2024-04-23T18:16:47.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:47 smithi037 ceph-mon[29404]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:47.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:47 smithi172 ceph-mon[32456]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:47.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:47 smithi173 ceph-mon[32906]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:49.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:49 smithi037 ceph-mon[29404]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:49.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:49 smithi172 ceph-mon[32456]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:49.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:49 smithi173 ceph-mon[32906]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:51.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:51 smithi037 ceph-mon[29404]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:51.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:51 smithi172 ceph-mon[32456]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:51.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:51 smithi173 ceph-mon[32906]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:52.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:52.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:52.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:52 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:52.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:52 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:52.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:52.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:52.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:52 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:53.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:53 smithi172 ceph-mon[32456]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:53.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:53 smithi037 ceph-mon[29404]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:53.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:53 smithi173 ceph-mon[32906]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.135+0000 7f410effd700 1 -- 172.21.15.37:0/3973001661 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4118062e80 con 0x7f41040a3c40 2024-04-23T18:16:55.130 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-04-23T18:16:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.137+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f41040a3c40 msgr2=0x7f41040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f41040a3c40 0x7f41040a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4114005950 tx=0x7f41140058e0 comp rx=0 tx=0).stop 2024-04-23T18:16:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 msgr2=0x7f4118074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 0x7f4118074e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f410800d950 tx=0x7f410800dc60 comp rx=0 tx=0).stop 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 shutdown_connections 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f41040a3c40 0x7f41040a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4118104640 0x7f4118075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4118102aa0 0x7f41180767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 --2- 172.21.15.37:0/3973001661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4118103ca0 0x7f4118074e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 >> 172.21.15.37:0/3973001661 conn(0x7f41180fe040 msgr2=0x7f4118107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 shutdown_connections 2024-04-23T18:16:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:16:55.138+0000 7f411fb7e700 1 -- 172.21.15.37:0/3973001661 wait complete. 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:55.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:55 smithi037 ceph-mon[29404]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:55.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:55 smithi172 ceph-mon[32456]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:16:55.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:55.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:55.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:55.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:55.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:55.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:55 smithi173 ceph-mon[32906]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:16:55.798 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.9.service 2024-04-23T18:16:55.801 INFO:tasks.cephadm:Deploying osd.10 on smithi172 with /dev/vg_nvme/lv_1... 2024-04-23T18:16:55.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:16:56.048 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:16:57.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:57 smithi037 ceph-mon[29404]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:57.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:57 smithi037 ceph-mon[29404]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:16:57.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:57 smithi037 ceph-mon[29404]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:16:57.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:57 smithi037 ceph-mon[29404]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:16:57.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:57 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:57.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:57 smithi172 ceph-mon[32456]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:57.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:57 smithi172 ceph-mon[32456]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:16:57.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:57 smithi172 ceph-mon[32456]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:16:57.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:57 smithi172 ceph-mon[32456]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:16:57.413 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:57 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:57.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:57 smithi173 ceph-mon[32906]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:57.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:57 smithi173 ceph-mon[32906]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:16:57.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:57 smithi173 ceph-mon[32906]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T18:16:57.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:57 smithi173 ceph-mon[32906]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-23T18:16:57.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:57 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:57.955 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:16:57.955 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:16:57.955 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-23T18:16:57.955 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-23T18:16:57.955 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011321 s, 926 MB/s 2024-04-23T18:16:57.955 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-23T18:16:58.328 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-mon[29404]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:16:58.329 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-mon[29404]: osdmap e68: 10 total, 9 up, 10 in 2024-04-23T18:16:58.329 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.329 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.329 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-mon[29404]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' 2024-04-23T18:16:58.329 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.329 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:16:58.030+0000 7f423d03f700 -1 osd.9 0 waiting for initial osdmap 2024-04-23T18:16:58.329 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:16:58 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:16:58.155+0000 7f4234e2a700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:16:58.516 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:58 smithi172 ceph-mon[32456]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:16:58.516 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:58 smithi172 ceph-mon[32456]: osdmap e68: 10 total, 9 up, 10 in 2024-04-23T18:16:58.516 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.516 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.516 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:58 smithi172 ceph-mon[32456]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' 2024-04-23T18:16:58.516 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.541 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-04-23T18:16:58.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:58 smithi173 ceph-mon[32906]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-23T18:16:58.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:58 smithi173 ceph-mon[32906]: osdmap e68: 10 total, 9 up, 10 in 2024-04-23T18:16:58.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:58 smithi173 ceph-mon[32906]: from='osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052]' entity='osd.9' 2024-04-23T18:16:58.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.203 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:59.204 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:59.204 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.204 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:16:59 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:59.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:59.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:16:59 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:59.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:16:59.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:16:59.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:16:59 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:17:00.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 -- 172.21.15.172:0/750093506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 msgr2=0x7fef54101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 --2- 172.21.15.172:0/750093506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54101260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef44009990 tx=0x7fef44009ca0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 -- 172.21.15.172:0/750093506 shutdown_connections 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 --2- 172.21.15.172:0/750093506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef541047c0 0x7fef5410b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 --2- 172.21.15.172:0/750093506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54101260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 --2- 172.21.15.172:0/750093506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 0x7fef54104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 -- 172.21.15.172:0/750093506 >> 172.21.15.172:0/750093506 conn(0x7fef540fc7a0 msgr2=0x7fef540febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 -- 172.21.15.172:0/750093506 shutdown_connections 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 -- 172.21.15.172:0/750093506 wait complete. 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 Processor -- start 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.286+0000 7fef5a20f700 1 -- start start 2024-04-23T18:17:00.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 0x7fef54101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef541047c0 0x7fef54102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef5410e730 con 0x7fef541047c0 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef5410e5b0 con 0x7fef54100e40 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef5410e8b0 con 0x7fef54103e20 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54103780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33330/0 (socket says 172.21.15.172:33330) 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 -- 172.21.15.172:0/3123864803 learned_addr learned my addr 172.21.15.172:0/3123864803 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef4bfff700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 0x7fef54101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef53fff700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef541047c0 0x7fef54102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 -- 172.21.15.172:0/3123864803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 msgr2=0x7fef54101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 0x7fef54101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 -- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef541047c0 msgr2=0x7fef54102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef541047c0 0x7fef54102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 -- 172.21.15.172:0/3123864803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef54102bc0 con 0x7fef54100e40 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef4bfff700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 0x7fef54101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef537fe700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54103780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b5c0 tx=0x7fef44009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44027070 con 0x7fef54100e40 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef44022470 con 0x7fef54100e40 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef541a2ed0 con 0x7fef54100e40 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.287+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4400f6c0 con 0x7fef54100e40 2024-04-23T18:17:00.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.288+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef541a32f0 con 0x7fef54100e40 2024-04-23T18:17:00.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.288+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef54069730 con 0x7fef54100e40 2024-04-23T18:17:00.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.290+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef4402a070 con 0x7fef54100e40 2024-04-23T18:17:00.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.290+0000 7fef517fa700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef400a3c40 0x7fef400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.290+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7fef440d0f90 con 0x7fef54100e40 2024-04-23T18:17:00.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.290+0000 7fef4bfff700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef400a3c40 0x7fef400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.291+0000 7fef4bfff700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef400a3c40 0x7fef400a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007900 tx=0x7fef3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:00.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.294+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef44013070 con 0x7fef54100e40 2024-04-23T18:17:00.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:00.499+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef54062f40 con 0x7fef400a3c40 2024-04-23T18:17:00.494 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:00 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi037 2024-04-23T18:17:00.494 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:00 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-23T18:17:00.494 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:00 smithi172 ceph-mon[32456]: osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] boot 2024-04-23T18:17:00.494 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:00 smithi172 ceph-mon[32456]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:17:00.494 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:17:00.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:00 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi037 2024-04-23T18:17:00.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:00 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-23T18:17:00.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:00 smithi173 ceph-mon[32906]: osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] boot 2024-04-23T18:17:00.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:00 smithi173 ceph-mon[32906]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:17:00.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:17:00.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:00 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi037 2024-04-23T18:17:00.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:00 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-23T18:17:00.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:00 smithi037 ceph-mon[29404]: osd.9 [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] boot 2024-04-23T18:17:00.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:00 smithi037 ceph-mon[29404]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T18:17:00.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:17:01.395 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:01 smithi172 ceph-mon[32456]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:17:01.395 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:01 smithi172 ceph-mon[32456]: osdmap e70: 10 total, 10 up, 10 in 2024-04-23T18:17:01.395 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:01 smithi172 ceph-mon[32456]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:01.395 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:01.395 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:01.395 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:01 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:01.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:01 smithi037 ceph-mon[29404]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:17:01.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:01 smithi037 ceph-mon[29404]: osdmap e70: 10 total, 10 up, 10 in 2024-04-23T18:17:01.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:01 smithi037 ceph-mon[29404]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:01.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:01.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:01.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:01 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:01.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:01 smithi173 ceph-mon[32906]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:17:01.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:01 smithi173 ceph-mon[32906]: osdmap e70: 10 total, 10 up, 10 in 2024-04-23T18:17:01.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:01 smithi173 ceph-mon[32906]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:01.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:01.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:01.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:01 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:03.511 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:03 smithi172 ceph-mon[32456]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:03.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:03 smithi173 ceph-mon[32906]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:03.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:03 smithi037 ceph-mon[29404]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:04.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:04 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/4201178222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]: dispatch 2024-04-23T18:17:04.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:04 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]: dispatch 2024-04-23T18:17:04.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:04 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]': finished 2024-04-23T18:17:04.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:04 smithi172 ceph-mon[32456]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:17:04.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:04 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:04.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:04 smithi172 ceph-mon[32456]: from='client.? 172.21.15.172:0/4115401086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:04.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:04 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/4201178222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]: dispatch 2024-04-23T18:17:04.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:04 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]: dispatch 2024-04-23T18:17:04.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:04 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]': finished 2024-04-23T18:17:04.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:04 smithi173 ceph-mon[32906]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:17:04.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:04 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:04.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:04 smithi173 ceph-mon[32906]: from='client.? 172.21.15.172:0/4115401086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:04.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:04 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/4201178222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]: dispatch 2024-04-23T18:17:04.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:04 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]: dispatch 2024-04-23T18:17:04.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:04 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72e96128-329b-4155-83cf-58bb44169b73"}]': finished 2024-04-23T18:17:04.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:04 smithi037 ceph-mon[29404]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:17:04.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:04 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:04.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:04 smithi037 ceph-mon[29404]: from='client.? 172.21.15.172:0/4115401086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:05.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:05 smithi172 ceph-mon[32456]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:05.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:05 smithi173 ceph-mon[32906]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:05.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:05 smithi037 ceph-mon[29404]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:07.439 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:07 smithi172 ceph-mon[32456]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:07.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:07 smithi173 ceph-mon[32906]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:07.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:07 smithi037 ceph-mon[29404]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:09.552 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:09 smithi172 ceph-mon[32456]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:09.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:09 smithi173 ceph-mon[32906]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:09.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:09 smithi037 ceph-mon[29404]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:11.465 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:11 smithi172 ceph-mon[32456]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:11.465 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:11 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:17:11.465 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:11 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:11.465 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:11 smithi172 ceph-mon[32456]: Deploying daemon osd.10 on smithi172 2024-04-23T18:17:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:11 smithi173 ceph-mon[32906]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:11 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:17:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:11 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:11.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:11 smithi173 ceph-mon[32906]: Deploying daemon osd.10 on smithi172 2024-04-23T18:17:11.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:11 smithi037 ceph-mon[29404]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:11.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:17:11.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:11.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:11 smithi037 ceph-mon[29404]: Deploying daemon osd.10 on smithi172 2024-04-23T18:17:13.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:13 smithi172 ceph-mon[32456]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:13.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:13 smithi173 ceph-mon[32906]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:13.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:13 smithi037 ceph-mon[29404]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:15.416 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:15 smithi172 ceph-mon[32456]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:15.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:15 smithi173 ceph-mon[32906]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:15.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:15 smithi037 ceph-mon[29404]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:16.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:16.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:16.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:16.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:16.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:16.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:16.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:16.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:16.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:17.500 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:17 smithi172 ceph-mon[32456]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:17.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:17 smithi173 ceph-mon[32906]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:17.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:17 smithi037 ceph-mon[29404]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:19.460 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 10 on host 'smithi172' 2024-04-23T18:17:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.466+0000 7fef517fa700 1 -- 172.21.15.172:0/3123864803 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fef54062f40 con 0x7fef400a3c40 2024-04-23T18:17:19.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.468+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef400a3c40 msgr2=0x7fef400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:19.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.468+0000 7fef5a20f700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef400a3c40 0x7fef400a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007900 tx=0x7fef3c008040 comp rx=0 tx=0).stop 2024-04-23T18:17:19.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 msgr2=0x7fef54103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54103780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b5c0 tx=0x7fef44009f90 comp rx=0 tx=0).stop 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 shutdown_connections 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef400a3c40 0x7fef400a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef541047c0 0x7fef54102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef54100e40 0x7fef54103780 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 --2- 172.21.15.172:0/3123864803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54103e20 0x7fef54101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 >> 172.21.15.172:0/3123864803 conn(0x7fef540fc7a0 msgr2=0x7fef54107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 shutdown_connections 2024-04-23T18:17:19.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-23T18:17:19.469+0000 7fef5a20f700 1 -- 172.21.15.172:0/3123864803 wait complete. 2024-04-23T18:17:20.014 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.015 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:19 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.040 DEBUG:teuthology.orchestra.run.smithi172:osd.10> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.10.service 2024-04-23T18:17:20.042 INFO:tasks.cephadm:Deploying osd.11 on smithi173 with /dev/vg_nvme/lv_1... 2024-04-23T18:17:20.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:17:20.068 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.069 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:19 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:19 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:20.287 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:17:21.020 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:20 smithi172 ceph-mon[32456]: from='osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:17:21.020 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:20 smithi172 ceph-mon[32456]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:17:21.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:20 smithi173 ceph-mon[32906]: from='osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:17:21.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:20 smithi173 ceph-mon[32906]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:17:21.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:20 smithi037 ceph-mon[29404]: from='osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:17:21.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:20 smithi037 ceph-mon[29404]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:17:22.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:21 smithi172 ceph-mon[32456]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:22.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:21 smithi172 ceph-mon[32456]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:17:22.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:21 smithi172 ceph-mon[32456]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:17:22.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:21 smithi172 ceph-mon[32456]: from='osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:17:22.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:21 smithi172 ceph-mon[32456]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:17:22.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:21 smithi173 ceph-mon[32906]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:22.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:21 smithi173 ceph-mon[32906]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:17:22.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:21 smithi173 ceph-mon[32906]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:17:22.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:21 smithi173 ceph-mon[32906]: from='osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:17:22.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:21 smithi173 ceph-mon[32906]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:17:22.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:21 smithi037 ceph-mon[29404]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:22.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:21 smithi037 ceph-mon[29404]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:17:22.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:21 smithi037 ceph-mon[29404]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T18:17:22.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:21 smithi037 ceph-mon[29404]: from='osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:17:22.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:21 smithi037 ceph-mon[29404]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113085 s, 927 MB/s 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 2024-04-23T18:17:22.239 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T18:17:22.749 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-mon[32456]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:17:22.750 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-mon[32456]: osdmap e73: 11 total, 10 up, 11 in 2024-04-23T18:17:22.750 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.750 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.750 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-mon[32456]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:22.750 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-mon[32456]: from='osd.10 ' entity='osd.10' 2024-04-23T18:17:22.750 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:17:22.584+0000 7f912f4eb700 -1 osd.10 0 waiting for initial osdmap 2024-04-23T18:17:22.750 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:17:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:17:22.755+0000 7f91272d6700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:17:22.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:22 smithi173 ceph-mon[32906]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:17:22.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:22 smithi173 ceph-mon[32906]: osdmap e73: 11 total, 10 up, 11 in 2024-04-23T18:17:22.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:22 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:22 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:22.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:22 smithi173 ceph-mon[32906]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:22.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:22 smithi173 ceph-mon[32906]: from='osd.10 ' entity='osd.10' 2024-04-23T18:17:23.006 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-04-23T18:17:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:22 smithi037 ceph-mon[29404]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-23T18:17:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:22 smithi037 ceph-mon[29404]: osdmap e73: 11 total, 10 up, 11 in 2024-04-23T18:17:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:22 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:22 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:22 smithi037 ceph-mon[29404]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:23.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:22 smithi037 ceph-mon[29404]: from='osd.10 ' entity='osd.10' 2024-04-23T18:17:23.286 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: osdmap e74: 11 total, 10 up, 11 in 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi172 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:23.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi172 to 4272M 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:23.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: osdmap e74: 11 total, 10 up, 11 in 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi172 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi172 to 4272M 2024-04-23T18:17:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: osdmap e74: 11 total, 10 up, 11 in 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi172 2024-04-23T18:17:24.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi172 to 4272M 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:24.123 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:24.123 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:24.123 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:24.820 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 -- 172.21.15.173:0/1932987084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb30102bb0 msgr2=0x7fdb30103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 --2- 172.21.15.173:0/1932987084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb30102bb0 0x7fdb30103010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009990 tx=0x7fdb20009ca0 comp rx=0 tx=0).stop 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 -- 172.21.15.173:0/1932987084 shutdown_connections 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 --2- 172.21.15.173:0/1932987084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb301019b0 0x7fdb30101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 --2- 172.21.15.173:0/1932987084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb30103550 0x7fdb30109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 --2- 172.21.15.173:0/1932987084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb30102bb0 0x7fdb30103010 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 -- 172.21.15.173:0/1932987084 >> 172.21.15.173:0/1932987084 conn(0x7fdb300fcf70 msgr2=0x7fdb300ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 -- 172.21.15.173:0/1932987084 shutdown_connections 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.824+0000 7fdb372cc700 1 -- 172.21.15.173:0/1932987084 wait complete. 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 Processor -- start 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 -- start start 2024-04-23T18:17:24.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 0x7fdb30198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30102bb0 0x7fdb301991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb30103550 0x7fdb301a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3010d450 con 0x7fdb30102bb0 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3010d2d0 con 0x7fdb30103550 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb372cc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3010d5d0 con 0x7fdb301019b0 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb35068700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 0x7fdb30198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb35068700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 0x7fdb30198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:59188/0 (socket says 172.21.15.173:59188) 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb35068700 1 -- 172.21.15.173:0/2657174651 learned_addr learned my addr 172.21.15.173:0/2657174651 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb34867700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30102bb0 0x7fdb301991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.825+0000 7fdb35869700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb30103550 0x7fdb301a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:24.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb35068700 1 -- 172.21.15.173:0/2657174651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb30103550 msgr2=0x7fdb301a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb35068700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb30103550 0x7fdb301a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb35068700 1 -- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30102bb0 msgr2=0x7fdb301991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb35068700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30102bb0 0x7fdb301991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb35068700 1 -- 172.21.15.173:0/2657174651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb301a0910 con 0x7fdb301019b0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb35068700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 0x7fdb30198c90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d980 tx=0x7fdb2c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c016070 con 0x7fdb301019b0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2c00fd40 con 0x7fdb301019b0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb301a0c00 con 0x7fdb301019b0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c019360 con 0x7fdb301019b0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.826+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301a1190 con 0x7fdb301019b0 2024-04-23T18:17:24.823 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.827+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb30069730 con 0x7fdb301019b0 2024-04-23T18:17:24.824 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.828+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb2c019070 con 0x7fdb301019b0 2024-04-23T18:17:24.824 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.829+0000 7fdb267fc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb1c0a3cc0 0x7fdb1c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:24.824 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.829+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7fdb2c0cf7c0 con 0x7fdb301019b0 2024-04-23T18:17:24.824 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.829+0000 7fdb34867700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb1c0a3cc0 0x7fdb1c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:24.825 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.830+0000 7fdb34867700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb1c0a3cc0 0x7fdb1c0a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb3019a1a0 tx=0x7fdb20005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:24.827 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:24.834+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb2c09c7d0 con 0x7fdb301019b0 2024-04-23T18:17:25.033 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:25.039+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdb3006b950 con 0x7fdb1c0a3cc0 2024-04-23T18:17:25.033 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:24 smithi173 ceph-mon[32906]: osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] boot 2024-04-23T18:17:25.033 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:24 smithi173 ceph-mon[32906]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:17:25.033 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:24 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:25.034 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:24 smithi173 ceph-mon[32906]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:25.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:24 smithi172 ceph-mon[32456]: osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] boot 2024-04-23T18:17:25.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:24 smithi172 ceph-mon[32456]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:17:25.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:24 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:25.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:24 smithi172 ceph-mon[32456]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:25.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:24 smithi037 ceph-mon[29404]: osd.10 [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] boot 2024-04-23T18:17:25.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:24 smithi037 ceph-mon[29404]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:17:25.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:24 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:17:25.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:24 smithi037 ceph-mon[29404]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:17:25.935 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:25 smithi173 ceph-mon[32906]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:17:25.936 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:25 smithi173 ceph-mon[32906]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:25.936 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:25 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:25.936 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:25 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:25.936 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:25 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:26.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:25 smithi172 ceph-mon[32456]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:17:26.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:25 smithi172 ceph-mon[32456]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:26.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:25 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:26.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:25 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:26.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:25 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:26.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:25 smithi037 ceph-mon[29404]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:17:26.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:25 smithi037 ceph-mon[29404]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:26.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:25 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:17:26.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:25 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:17:26.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:25 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:26.928 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:26 smithi173 ceph-mon[32906]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:17:26.928 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:26 smithi173 ceph-mon[32906]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:27.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:26 smithi172 ceph-mon[32456]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:17:27.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:26 smithi172 ceph-mon[32456]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:27.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:26 smithi037 ceph-mon[29404]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T18:17:27.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:26 smithi037 ceph-mon[29404]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:28.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:27 smithi172 ceph-mon[32456]: osdmap e78: 11 total, 11 up, 11 in 2024-04-23T18:17:28.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:27 smithi173 ceph-mon[32906]: osdmap e78: 11 total, 11 up, 11 in 2024-04-23T18:17:28.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:27 smithi037 ceph-mon[29404]: osdmap e78: 11 total, 11 up, 11 in 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: osdmap e79: 11 total, 11 up, 11 in 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/1785407592' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]: dispatch 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]: dispatch 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]': finished 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:29.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:28 smithi172 ceph-mon[32456]: from='client.? 172.21.15.173:0/2685772732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:29.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: osdmap e79: 11 total, 11 up, 11 in 2024-04-23T18:17:29.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/1785407592' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]: dispatch 2024-04-23T18:17:29.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]: dispatch 2024-04-23T18:17:29.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]': finished 2024-04-23T18:17:29.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:17:29.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:29.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:29.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:28 smithi173 ceph-mon[32906]: from='client.? 172.21.15.173:0/2685772732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:29.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: osdmap e79: 11 total, 11 up, 11 in 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/1785407592' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]: dispatch 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]: dispatch 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d918711-72d6-4a04-97cc-4fe8dad4a3e8"}]': finished 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:29.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:28 smithi037 ceph-mon[29404]: from='client.? 172.21.15.173:0/2685772732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:17:31.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:30 smithi173 ceph-mon[32906]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-23T18:17:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:30 smithi037 ceph-mon[29404]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-23T18:17:31.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:30 smithi172 ceph-mon[32456]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-23T18:17:33.068 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:32 smithi173 ceph-mon[32906]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:17:33.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:32 smithi037 ceph-mon[29404]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:17:33.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:32 smithi172 ceph-mon[32456]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:17:35.071 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:34 smithi173 ceph-mon[32906]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-23T18:17:35.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:34 smithi037 ceph-mon[29404]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-23T18:17:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:34 smithi172 ceph-mon[32456]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-23T18:17:36.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:17:36.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:36.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:35 smithi037 ceph-mon[29404]: Deploying daemon osd.11 on smithi173 2024-04-23T18:17:36.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:17:36.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:36.125 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:35 smithi173 ceph-mon[32906]: Deploying daemon osd.11 on smithi173 2024-04-23T18:17:36.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:17:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:35 smithi172 ceph-mon[32456]: Deploying daemon osd.11 on smithi173 2024-04-23T18:17:36.977 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:36 smithi173 ceph-mon[32906]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-23T18:17:37.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:36 smithi037 ceph-mon[29404]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-23T18:17:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:36 smithi172 ceph-mon[32456]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-23T18:17:39.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:38 smithi173 ceph-mon[32906]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:17:39.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:38 smithi037 ceph-mon[29404]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:17:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:38 smithi172 ceph-mon[32456]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:17:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:40 smithi173 ceph-mon[32906]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:17:41.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:40 smithi037 ceph-mon[29404]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:17:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:40 smithi172 ceph-mon[32456]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T18:17:42.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:42.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:42.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:42.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:42.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:42.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:42.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:42.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:42.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:43.281 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:42 smithi173 ceph-mon[32906]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:43.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:42 smithi172 ceph-mon[32456]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:43.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:42 smithi037 ceph-mon[29404]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:44.955 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.961+0000 7fdb267fc700 1 -- 172.21.15.173:0/2657174651 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdb3006b950 con 0x7fdb1c0a3cc0 2024-04-23T18:17:44.955 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 11 on host 'smithi173' 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb1c0a3cc0 msgr2=0x7fdb1c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb1c0a3cc0 0x7fdb1c0a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb3019a1a0 tx=0x7fdb20005fb0 comp rx=0 tx=0).stop 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 msgr2=0x7fdb30198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 0x7fdb30198c90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d980 tx=0x7fdb2c00dc90 comp rx=0 tx=0).stop 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 shutdown_connections 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb1c0a3cc0 0x7fdb1c0a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb30103550 0x7fdb301a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:44.956 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb301019b0 0x7fdb30198c90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:44.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 --2- 172.21.15.173:0/2657174651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30102bb0 0x7fdb301991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:44.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 >> 172.21.15.173:0/2657174651 conn(0x7fdb300fcf70 msgr2=0x7fdb3006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:44.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 shutdown_connections 2024-04-23T18:17:44.957 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:44.962+0000 7fdb372cc700 1 -- 172.21.15.173:0/2657174651 wait complete. 2024-04-23T18:17:45.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:45.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:45.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:44 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:44 smithi172 ceph-mon[32456]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:44 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:44 smithi173 ceph-mon[32906]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:45.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:45.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:45.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:44 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:44 smithi037 ceph-mon[29404]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:45.641 DEBUG:teuthology.orchestra.run.smithi173:osd.11> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.11.service 2024-04-23T18:17:45.643 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-23T18:17:45.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T18:17:45.899 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:45 smithi173 ceph-mon[32906]: from='osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:17:45.899 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:45 smithi173 ceph-mon[32906]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:17:45.899 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:45 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.899 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:45 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:45.899 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:45 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:45.905 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:45 smithi037 ceph-mon[29404]: from='osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:17:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:45 smithi037 ceph-mon[29404]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:17:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:45 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:45 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:46.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:45 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:46.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:45 smithi172 ceph-mon[32456]: from='osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:17:46.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:45 smithi172 ceph-mon[32456]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:17:46.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:45 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:46.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:45 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:46.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:45 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:46.920 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-mon[32906]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:17:46.920 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-mon[32906]: osdmap e81: 12 total, 11 up, 12 in 2024-04-23T18:17:46.920 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:46.920 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-mon[32906]: from='osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:17:46.920 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-mon[32906]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:17:46.920 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-mon[32906]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:46.921 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:17:46.797+0000 7f8178dbd700 -1 osd.11 0 waiting for initial osdmap 2024-04-23T18:17:46.921 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:17:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:17:46.815+0000 7f8174bb0700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:17:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:46 smithi172 ceph-mon[32456]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:17:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:46 smithi172 ceph-mon[32456]: osdmap e81: 12 total, 11 up, 12 in 2024-04-23T18:17:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:46 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:46 smithi172 ceph-mon[32456]: from='osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:17:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:46 smithi172 ceph-mon[32456]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:17:47.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:46 smithi172 ceph-mon[32456]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:47.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:46 smithi037 ceph-mon[29404]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:17:47.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:46 smithi037 ceph-mon[29404]: osdmap e81: 12 total, 11 up, 12 in 2024-04-23T18:17:47.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:46 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:47.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:46 smithi037 ceph-mon[29404]: from='osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:17:47.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:46 smithi037 ceph-mon[29404]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T18:17:47.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:46 smithi037 ceph-mon[29404]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:47.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3765990321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d700754a0 msgr2=0x7f9d70075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3765990321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d700754a0 0x7f9d70075920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0098f0 tx=0x7f9d5c009c00 comp rx=0 tx=0).stop 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3765990321 shutdown_connections 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3765990321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d7010f0a0 0x7f9d70111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3765990321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d70076a70 0x7f9d70074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3765990321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d700754a0 0x7f9d70075920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3765990321 >> 172.21.15.37:0/3765990321 conn(0x7f9d700fe060 msgr2=0x7f9d701004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3765990321 shutdown_connections 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3765990321 wait complete. 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 Processor -- start 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.701+0000 7f9d76ab5700 1 -- start start 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d76ab5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d700754a0 0x7f9d7019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d76ab5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d70076a70 0x7f9d7019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:47.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d76ab5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7010f0a0 0x7f9d701a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d76ab5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70114b70 con 0x7f9d7010f0a0 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d76ab5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701149f0 con 0x7f9d700754a0 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d76ab5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70114cf0 con 0x7f9d70076a70 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d75052700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7010f0a0 0x7f9d701a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d700754a0 0x7f9d7019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d75052700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7010f0a0 0x7f9d701a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55218/0 (socket says 172.21.15.37:55218) 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d75052700 1 -- 172.21.15.37:0/3850215825 learned_addr learned my addr 172.21.15.37:0/3850215825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d6ffff700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d70076a70 0x7f9d7019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 -- 172.21.15.37:0/3850215825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d70076a70 msgr2=0x7f9d7019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d70076a70 0x7f9d7019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 -- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7010f0a0 msgr2=0x7f9d701a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7010f0a0 0x7f9d701a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 -- 172.21.15.37:0/3850215825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d701a6ec0 con 0x7f9d700754a0 2024-04-23T18:17:47.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d6ffff700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d70076a70 0x7f9d7019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:17:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.702+0000 7f9d74851700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d700754a0 0x7f9d7019f260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400e9c0 tx=0x7f9d6400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.703+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400c9c0 con 0x7f9d700754a0 2024-04-23T18:17:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.703+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a71b0 con 0x7f9d700754a0 2024-04-23T18:17:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.703+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6400cb60 con 0x7f9d700754a0 2024-04-23T18:17:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.703+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64018680 con 0x7f9d700754a0 2024-04-23T18:17:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.703+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a7740 con 0x7f9d700754a0 2024-04-23T18:17:47.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.704+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d70069730 con 0x7f9d700754a0 2024-04-23T18:17:47.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.705+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d64010430 con 0x7f9d700754a0 2024-04-23T18:17:47.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.706+0000 7f9d6dffb700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:47.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.706+0000 7f9d6ffff700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:47.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.706+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6847+0+0 (secure 0 0 0) 0x7f9d640d0340 con 0x7f9d700754a0 2024-04-23T18:17:47.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.706+0000 7f9d6ffff700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9d600a3c40 0x7f9d600a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009fd0 tx=0x7f9d5c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:47.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.710+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d6409d240 con 0x7f9d700754a0 2024-04-23T18:17:47.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.924+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9d70063650 con 0x7f9d700754a0 2024-04-23T18:17:47.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.925+0000 7f9d6dffb700 1 -- 172.21.15.37:0/3850215825 <== mon.2 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f9d6409cb40 con 0x7f9d700754a0 2024-04-23T18:17:47.918 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:17:47.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9d600a3c40 msgr2=0x7f9d600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:47.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9d600a3c40 0x7f9d600a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009fd0 tx=0x7f9d5c019040 comp rx=0 tx=0).stop 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d700754a0 msgr2=0x7f9d7019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d700754a0 0x7f9d7019f260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400e9c0 tx=0x7f9d6400ecd0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 shutdown_connections 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9d600a3c40 0x7f9d600a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7010f0a0 0x7f9d701a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d700754a0 0x7f9d7019f260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 --2- 172.21.15.37:0/3850215825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d70076a70 0x7f9d7019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 >> 172.21.15.37:0/3850215825 conn(0x7f9d700fe060 msgr2=0x7f9d70102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 shutdown_connections 2024-04-23T18:17:47.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:47.927+0000 7f9d76ab5700 1 -- 172.21.15.37:0/3850215825 wait complete. 2024-04-23T18:17:48.054 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:47 smithi173 ceph-mon[32906]: purged_snaps scrub starts 2024-04-23T18:17:48.055 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:47 smithi173 ceph-mon[32906]: purged_snaps scrub ok 2024-04-23T18:17:48.055 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:47 smithi173 ceph-mon[32906]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:17:48.055 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:47 smithi173 ceph-mon[32906]: osdmap e82: 12 total, 11 up, 12 in 2024-04-23T18:17:48.055 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:47 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.055 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:47 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:47 smithi172 ceph-mon[32456]: purged_snaps scrub starts 2024-04-23T18:17:48.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:47 smithi172 ceph-mon[32456]: purged_snaps scrub ok 2024-04-23T18:17:48.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:47 smithi172 ceph-mon[32456]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:17:48.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:47 smithi172 ceph-mon[32456]: osdmap e82: 12 total, 11 up, 12 in 2024-04-23T18:17:48.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:47 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:47 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:47 smithi037 ceph-mon[29404]: purged_snaps scrub starts 2024-04-23T18:17:48.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:47 smithi037 ceph-mon[29404]: purged_snaps scrub ok 2024-04-23T18:17:48.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:47 smithi037 ceph-mon[29404]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T18:17:48.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:47 smithi037 ceph-mon[29404]: osdmap e82: 12 total, 11 up, 12 in 2024-04-23T18:17:48.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:47 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:47 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.509 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1713896267,"num_in_osds":12,"osd_in_since":1713896247,"num_remapped_pgs":0} 2024-04-23T18:17:48.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:17:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:48.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] boot 2024-04-23T18:17:48.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:17:48.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3850215825' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:48.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:48 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] boot 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3850215825' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:48 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: osd.11 [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] boot 2024-04-23T18:17:49.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:17:49.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:17:49.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:17:49.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3850215825' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:49.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:48 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:49 smithi037 ceph-mon[29404]: Detected new or changed devices on smithi173 2024-04-23T18:17:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:49 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-23T18:17:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:49 smithi037 ceph-mon[29404]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:17:49.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:49 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:50.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:49 smithi172 ceph-mon[32456]: Detected new or changed devices on smithi173 2024-04-23T18:17:50.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:49 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-23T18:17:50.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:50.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:50.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:49 smithi172 ceph-mon[32456]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:17:50.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:49 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:50.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:49 smithi173 ceph-mon[32906]: Detected new or changed devices on smithi173 2024-04-23T18:17:50.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:49 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-23T18:17:50.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:50.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:50.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:49 smithi173 ceph-mon[32906]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:17:50.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:49 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:50.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.424+0000 7f088155a700 1 -- 172.21.15.37:0/2796491720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06b0f0 msgr2=0x7f087c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:50.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.424+0000 7f088155a700 1 --2- 172.21.15.37:0/2796491720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06b0f0 0x7f087c10f900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0874009f90 tx=0x7f087400b330 comp rx=0 tx=0).stop 2024-04-23T18:17:50.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 -- 172.21.15.37:0/2796491720 shutdown_connections 2024-04-23T18:17:50.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 --2- 172.21.15.37:0/2796491720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06b0f0 0x7f087c10f900 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 --2- 172.21.15.37:0/2796491720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c108a30 0x7f087c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 --2- 172.21.15.37:0/2796491720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f087c06a750 0x7f087c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 -- 172.21.15.37:0/2796491720 >> 172.21.15.37:0/2796491720 conn(0x7f087c0788f0 msgr2=0x7f087c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:50.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 -- 172.21.15.37:0/2796491720 shutdown_connections 2024-04-23T18:17:50.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 -- 172.21.15.37:0/2796491720 wait complete. 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.425+0000 7f088155a700 1 Processor -- start 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 -- start start 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 0x7f087c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c06b0f0 0x7f087c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f087c108a30 0x7f087c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f087c112bd0 con 0x7f087c06a750 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f087c112a50 con 0x7f087c108a30 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f088155a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f087c112d50 con 0x7f087c06b0f0 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 0x7f087c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 0x7f087c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55228/0 (socket says 172.21.15.37:55228) 2024-04-23T18:17:50.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087a7fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c06b0f0 0x7f087c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087affd700 1 -- 172.21.15.37:0/35771701 learned_addr learned my addr 172.21.15.37:0/35771701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087b7fe700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f087c108a30 0x7f087c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087a7fc700 1 -- 172.21.15.37:0/35771701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f087c108a30 msgr2=0x7f087c19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087a7fc700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f087c108a30 0x7f087c19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087a7fc700 1 -- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 msgr2=0x7f087c199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087a7fc700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 0x7f087c199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087a7fc700 1 -- 172.21.15.37:0/35771701 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c19e090 con 0x7f087c06b0f0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.426+0000 7f087affd700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 0x7f087c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.427+0000 7f087a7fc700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c06b0f0 0x7f087c1995c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f087000d950 tx=0x7f087000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.427+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870016070 con 0x7f087c06b0f0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.427+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087c19e380 con 0x7f087c06b0f0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.427+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087000fd40 con 0x7f087c06b0f0 2024-04-23T18:17:50.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.427+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087000e3b0 con 0x7f087c06b0f0 2024-04-23T18:17:50.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.427+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087c1a5c00 con 0x7f087c06b0f0 2024-04-23T18:17:50.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.428+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087c069730 con 0x7f087c06b0f0 2024-04-23T18:17:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.429+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0870019070 con 0x7f087c06b0f0 2024-04-23T18:17:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.430+0000 7f0863fff700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f08640a3c40 0x7f08640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.430+0000 7f087affd700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f08640a3c40 0x7f08640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.430+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f08700cff10 con 0x7f087c06b0f0 2024-04-23T18:17:50.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.431+0000 7f087affd700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f08640a3c40 0x7f08640a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f086c011550 tx=0x7f086c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:50.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.433+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f087009ccf0 con 0x7f087c06b0f0 2024-04-23T18:17:50.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.642+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f087c02df80 con 0x7f087c06b0f0 2024-04-23T18:17:50.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.644+0000 7f0863fff700 1 -- 172.21.15.37:0/35771701 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18480 (secure 0 0 0) 0x7f087009c5f0 con 0x7f087c06b0f0 2024-04-23T18:17:50.638 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:17:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:17:48.804072+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f08640a3c40 msgr2=0x7f08640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f08640a3c40 0x7f08640a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f086c011550 tx=0x7f086c009310 comp rx=0 tx=0).stop 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c06b0f0 msgr2=0x7f087c1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c06b0f0 0x7f087c1995c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f087000d950 tx=0x7f087000dc60 comp rx=0 tx=0).stop 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 shutdown_connections 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f08640a3c40 0x7f08640a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c06a750 0x7f087c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f087c108a30 0x7f087c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 --2- 172.21.15.37:0/35771701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f087c06b0f0 0x7f087c1995c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.646+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 >> 172.21.15.37:0/35771701 conn(0x7f087c0788f0 msgr2=0x7f087c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:50.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.647+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 shutdown_connections 2024-04-23T18:17:50.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:50.647+0000 7f088155a700 1 -- 172.21.15.37:0/35771701 wait complete. 2024-04-23T18:17:51.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:50 smithi172 ceph-mon[32456]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:51.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:50 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/35771701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:17:51.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:50 smithi173 ceph-mon[32906]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:51.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:50 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/35771701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:17:51.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:50 smithi037 ceph-mon[29404]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:51.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:50 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/35771701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:17:51.305 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:17:51.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T18:17:51.552 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:53.115 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:52 smithi037 ceph-mon[29404]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:53.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 -- 172.21.15.37:0/4047297010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28406b0f0 msgr2=0x7fc28410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:53.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 --2- 172.21.15.37:0/4047297010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28406b0f0 0x7fc28410f900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b0a0 tx=0x7fc27c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 -- 172.21.15.37:0/4047297010 shutdown_connections 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 --2- 172.21.15.37:0/4047297010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28406b0f0 0x7fc28410f900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 --2- 172.21.15.37:0/4047297010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284108a30 0x7fc284108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 --2- 172.21.15.37:0/4047297010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28406a750 0x7fc28406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 -- 172.21.15.37:0/4047297010 >> 172.21.15.37:0/4047297010 conn(0x7fc2840788f0 msgr2=0x7fc284078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 -- 172.21.15.37:0/4047297010 shutdown_connections 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.121+0000 7fc28a971700 1 -- 172.21.15.37:0/4047297010 wait complete. 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.122+0000 7fc28a971700 1 Processor -- start 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.122+0000 7fc28a971700 1 -- start start 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc28a971700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28406a750 0x7fc284199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc28a971700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc28406b0f0 0x7fc284199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc28a971700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 0x7fc28419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc28a971700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc284112c20 con 0x7fc284108a30 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc28a971700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc284112aa0 con 0x7fc28406a750 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc28a971700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc284112da0 con 0x7fc28406b0f0 2024-04-23T18:17:53.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 0x7fc28419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 0x7fc28419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55240/0 (socket says 172.21.15.37:55240) 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 -- 172.21.15.37:0/758724780 learned_addr learned my addr 172.21.15.37:0/758724780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc283fff700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28406a750 0x7fc284199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc2837fe700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc28406b0f0 0x7fc284199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 -- 172.21.15.37:0/758724780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc28406b0f0 msgr2=0x7fc284199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc28406b0f0 0x7fc284199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 -- 172.21.15.37:0/758724780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28406a750 msgr2=0x7fc284199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28406a750 0x7fc284199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 -- 172.21.15.37:0/758724780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28419e140 con 0x7fc284108a30 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.123+0000 7fc288f0e700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 0x7fc28419da00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009fd0 tx=0x7fc27c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.124+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c021070 con 0x7fc284108a30 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.124+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28419e3d0 con 0x7fc284108a30 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.124+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c007c00 con 0x7fc284108a30 2024-04-23T18:17:53.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.124+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0052a0 con 0x7fc284108a30 2024-04-23T18:17:53.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.124+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2841a5cb0 con 0x7fc284108a30 2024-04-23T18:17:53.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.125+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc284069730 con 0x7fc284108a30 2024-04-23T18:17:53.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.126+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc27c024070 con 0x7fc284108a30 2024-04-23T18:17:53.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.127+0000 7fc2817fa700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc26c0a3c40 0x7fc26c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:53.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.127+0000 7fc283fff700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc26c0a3c40 0x7fc26c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.127+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc27c00f020 con 0x7fc284108a30 2024-04-23T18:17:53.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.127+0000 7fc283fff700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc26c0a3c40 0x7fc26c0a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc274005950 tx=0x7fc27400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:53.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.131+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc27c02b030 con 0x7fc284108a30 2024-04-23T18:17:53.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.337+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc28419a490 con 0x7fc284108a30 2024-04-23T18:17:53.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.337+0000 7fc2817fa700 1 -- 172.21.15.37:0/758724780 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fc27c024e40 con 0x7fc284108a30 2024-04-23T18:17:53.331 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.339+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc26c0a3c40 msgr2=0x7fc26c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.339+0000 7fc28a971700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc26c0a3c40 0x7fc26c0a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc274005950 tx=0x7fc27400b410 comp rx=0 tx=0).stop 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.339+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 msgr2=0x7fc28419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.339+0000 7fc28a971700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 0x7fc28419da00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009fd0 tx=0x7fc27c004dc0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 shutdown_connections 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc26c0a3c40 0x7fc26c0a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc284108a30 0x7fc28419da00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28406a750 0x7fc284199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 --2- 172.21.15.37:0/758724780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc28406b0f0 0x7fc284199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 >> 172.21.15.37:0/758724780 conn(0x7fc2840788f0 msgr2=0x7fc2840ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 shutdown_connections 2024-04-23T18:17:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:53.340+0000 7fc28a971700 1 -- 172.21.15.37:0/758724780 wait complete. 2024-04-23T18:17:53.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:52 smithi172 ceph-mon[32456]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:53.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:52 smithi173 ceph-mon[32906]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:53.930 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:53 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/758724780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.b on smithi172 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.e on smithi172 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.h on smithi172 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.k on smithi172 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.c on smithi173 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.f on smithi173 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.i on smithi173 2024-04-23T18:17:53.932 INFO:tasks.cephadm:Adding mds.l on smithi173 2024-04-23T18:17:53.933 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l' 2024-04-23T18:17:54.179 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:17:54.296 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:53 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/758724780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:17:54.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:53 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/758724780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:17:55.299 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:54 smithi173 ceph-mon[32906]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:55.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:54 smithi172 ceph-mon[32456]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:55.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:54 smithi037 ceph-mon[29404]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 -- 172.21.15.173:0/2721733778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0102ba0 msgr2=0x7f57b0103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 --2- 172.21.15.173:0/2721733778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0102ba0 0x7f57b0103000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009990 tx=0x7f57a0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 -- 172.21.15.173:0/2721733778 shutdown_connections 2024-04-23T18:17:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 --2- 172.21.15.173:0/2721733778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b01019a0 0x7f57b0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 --2- 172.21.15.173:0/2721733778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0103540 0x7f57b0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.752 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 --2- 172.21.15.173:0/2721733778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0102ba0 0x7f57b0103000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 -- 172.21.15.173:0/2721733778 >> 172.21.15.173:0/2721733778 conn(0x7f57b00fcf40 msgr2=0x7f57b00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 -- 172.21.15.173:0/2721733778 shutdown_connections 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 -- 172.21.15.173:0/2721733778 wait complete. 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 Processor -- start 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.756+0000 7f57b87aa700 1 -- start start 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b01019a0 0x7f57b0105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0102ba0 0x7f57b0103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 0x7f57b01044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57b010d4b0 con 0x7f57b01019a0 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f57b010d330 con 0x7f57b0102ba0 2024-04-23T18:17:55.753 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57b010d630 con 0x7f57b0103540 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 0x7f57b01044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 0x7f57b01044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:54102/0 (socket says 172.21.15.173:54102) 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 -- 172.21.15.173:0/3648020287 learned_addr learned my addr 172.21.15.173:0/3648020287 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6546700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b01019a0 0x7f57b0105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b5d45700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0102ba0 0x7f57b0103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 -- 172.21.15.173:0/3648020287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0102ba0 msgr2=0x7f57b0103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0102ba0 0x7f57b0103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 -- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b01019a0 msgr2=0x7f57b0105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b01019a0 0x7f57b0105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.754 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 -- 172.21.15.173:0/3648020287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b0104d70 con 0x7f57b0103540 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b5d45700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0102ba0 0x7f57b0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b6d47700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 0x7f57b01044c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ada0 tx=0x7f57a800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8017020 con 0x7f57b0103540 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a800ce30 con 0x7f57b0103540 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.757+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b01a2e40 con 0x7f57b0103540 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.758+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8005350 con 0x7f57b0103540 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.758+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b01a32f0 con 0x7f57b0103540 2024-04-23T18:17:55.755 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.758+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b0069730 con 0x7f57b0103540 2024-04-23T18:17:55.756 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.759+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f57a800c980 con 0x7f57b0103540 2024-04-23T18:17:55.756 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.760+0000 7f57a77fe700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f579c0a3cc0 0x7f579c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:55.756 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.760+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f57a80cf5b0 con 0x7f57b0103540 2024-04-23T18:17:55.756 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.760+0000 7f57b6546700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f579c0a3cc0 0x7f579c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:55.756 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.761+0000 7f57b6546700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f579c0a3cc0 0x7f579c0a6180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f57b0102a00 tx=0x7f57ac009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:55.758 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.765+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57a8021020 con 0x7f57b0103540 2024-04-23T18:17:55.971 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.978+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}) -- 0x7f57b006b980 con 0x7f579c0a3cc0 2024-04-23T18:17:55.979 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.985+0000 7f57a77fe700 1 -- 172.21.15.173:0/3648020287 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f57b006b980 con 0x7f579c0a3cc0 2024-04-23T18:17:55.979 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mds.all update... 2024-04-23T18:17:55.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f579c0a3cc0 msgr2=0x7f579c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f579c0a3cc0 0x7f579c0a6180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f57b0102a00 tx=0x7f57ac009190 comp rx=0 tx=0).stop 2024-04-23T18:17:55.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 msgr2=0x7f57b01044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 0x7f57b01044c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ada0 tx=0x7f57a800c330 comp rx=0 tx=0).stop 2024-04-23T18:17:55.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 shutdown_connections 2024-04-23T18:17:55.982 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f579c0a3cc0 0x7f579c0a6180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b01019a0 0x7f57b0105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57b0103540 0x7f57b01044c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 --2- 172.21.15.173:0/3648020287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57b0102ba0 0x7f57b0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 >> 172.21.15.173:0/3648020287 conn(0x7f57b00fcf40 msgr2=0x7f57b006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:55.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 shutdown_connections 2024-04-23T18:17:55.983 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T18:17:55.988+0000 7f57b87aa700 1 -- 172.21.15.173:0/3648020287 wait complete. 2024-04-23T18:17:56.682 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.a.service 2024-04-23T18:17:56.684 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.d.service 2024-04-23T18:17:56.727 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.g.service 2024-04-23T18:17:56.769 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.j.service 2024-04-23T18:17:56.812 DEBUG:teuthology.orchestra.run.smithi172:mds.b> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.b.service 2024-04-23T18:17:56.814 DEBUG:teuthology.orchestra.run.smithi172:mds.e> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.e.service 2024-04-23T18:17:56.857 DEBUG:teuthology.orchestra.run.smithi172:mds.h> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.h.service 2024-04-23T18:17:56.900 DEBUG:teuthology.orchestra.run.smithi172:mds.k> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.k.service 2024-04-23T18:17:56.943 DEBUG:teuthology.orchestra.run.smithi173:mds.c> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.c.service 2024-04-23T18:17:56.945 DEBUG:teuthology.orchestra.run.smithi173:mds.f> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.f.service 2024-04-23T18:17:56.989 DEBUG:teuthology.orchestra.run.smithi173:mds.i> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.i.service 2024-04-23T18:17:57.033 DEBUG:teuthology.orchestra.run.smithi173:mds.l> sudo journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.l.service 2024-04-23T18:17:57.078 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-23T18:17:57.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:57.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:56 smithi172 ceph-mon[32456]: Deploying daemon mds.l on smithi173 2024-04-23T18:17:57.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 -- 172.21.15.37:0/2145598409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0401031b0 msgr2=0x7fb040103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 --2- 172.21.15.37:0/2145598409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0401031b0 0x7fb040103630 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009900 tx=0x7fb02c02a410 comp rx=0 tx=0).stop 2024-04-23T18:17:57.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 -- 172.21.15.37:0/2145598409 shutdown_connections 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 --2- 172.21.15.37:0/2145598409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb040103b70 0x7fb04010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 --2- 172.21.15.37:0/2145598409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0401031b0 0x7fb040103630 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 --2- 172.21.15.37:0/2145598409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb040101fc0 0x7fb0401023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 -- 172.21.15.37:0/2145598409 >> 172.21.15.37:0/2145598409 conn(0x7fb0400fd550 msgr2=0x7fb0400ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 -- 172.21.15.37:0/2145598409 shutdown_connections 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.219+0000 7fb046e10640 1 -- 172.21.15.37:0/2145598409 wait complete. 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 Processor -- start 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 -- start start 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 0x7fb040198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0401031b0 0x7fb0401987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb040103b70 0x7fb04019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb04010db00 con 0x7fb040101fc0 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb04010d980 con 0x7fb0401031b0 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb046e10640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb04010dc80 con 0x7fb040103b70 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 0x7fb040198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 0x7fb040198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34904/0 (socket says 172.21.15.37:34904) 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 -- 172.21.15.37:0/3484045335 learned_addr learned my addr 172.21.15.37:0/3484045335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb0453ad640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb040103b70 0x7fb04019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb03ffff640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0401031b0 0x7fb0401987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 -- 172.21.15.37:0/3484045335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb040103b70 msgr2=0x7fb04019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb040103b70 0x7fb04019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 -- 172.21.15.37:0/3484045335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0401031b0 msgr2=0x7fb0401987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0401031b0 0x7fb0401987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.220+0000 7fb044bac640 1 -- 172.21.15.37:0/3484045335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04019fe00 con 0x7fb040101fc0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.221+0000 7fb044bac640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 0x7fb040198280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb03400e9c0 tx=0x7fb03400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.221+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03400cd80 con 0x7fb040101fc0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.221+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0340035f0 con 0x7fb040101fc0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.221+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0401a00f0 con 0x7fb040101fc0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.221+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034010690 con 0x7fb040101fc0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.221+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0401a05f0 con 0x7fb040101fc0 2024-04-23T18:17:57.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.222+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb040109300 con 0x7fb040101fc0 2024-04-23T18:17:57.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.223+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb034004000 con 0x7fb040101fc0 2024-04-23T18:17:57.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.224+0000 7fb03dffb640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb0300a3bb0 0x7fb0300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.224+0000 7fb03ffff640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb0300a3bb0 0x7fb0300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.224+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb0340d05a0 con 0x7fb040101fc0 2024-04-23T18:17:57.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.224+0000 7fb03ffff640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb0300a3bb0 0x7fb0300a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0401996f0 tx=0x7fb02c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.228+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb03409d380 con 0x7fb040101fc0 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:57.284 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:17:57.285 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:17:57.285 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:57.285 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:56 smithi173 ceph-mon[32906]: Deploying daemon mds.l on smithi173 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:17:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:57.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:56 smithi037 ceph-mon[29404]: Deploying daemon mds.l on smithi173 2024-04-23T18:17:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.418+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb04002d5c0 con 0x7fb040101fc0 2024-04-23T18:17:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.419+0000 7fb03dffb640 1 -- 172.21.15.37:0/3484045335 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18480 (secure 0 0 0) 0x7fb03409cc80 con 0x7fb040101fc0 2024-04-23T18:17:57.413 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:17:57.414 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:17:48.804072+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:17:57.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb0300a3bb0 msgr2=0x7fb0300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb0300a3bb0 0x7fb0300a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0401996f0 tx=0x7fb02c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 msgr2=0x7fb040198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 0x7fb040198280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb03400e9c0 tx=0x7fb03400ee80 comp rx=0 tx=0).stop 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 shutdown_connections 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb0300a3bb0 0x7fb0300a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb040103b70 0x7fb04019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0401031b0 0x7fb0401987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 --2- 172.21.15.37:0/3484045335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb040101fc0 0x7fb040198280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 >> 172.21.15.37:0/3484045335 conn(0x7fb0400fd550 msgr2=0x7fb0401064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 shutdown_connections 2024-04-23T18:17:57.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.421+0000 7fb046e10640 1 -- 172.21.15.37:0/3484045335 wait complete. 2024-04-23T18:17:57.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:17:57.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:17:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 -- 172.21.15.37:0/1448524807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 msgr2=0x7fcbf8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/1448524807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8101280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009900 tx=0x7fcbe802a410 comp rx=0 tx=0).stop 2024-04-23T18:17:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 -- 172.21.15.37:0/1448524807 shutdown_connections 2024-04-23T18:17:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/1448524807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8102a00 0x7fcbf8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/1448524807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbf8102040 0x7fcbf81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/1448524807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8101280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 -- 172.21.15.37:0/1448524807 >> 172.21.15.37:0/1448524807 conn(0x7fcbf80fc440 msgr2=0x7fcbf80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 -- 172.21.15.37:0/1448524807 shutdown_connections 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 -- 172.21.15.37:0/1448524807 wait complete. 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 Processor -- start 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.538+0000 7fcbfeb95640 1 -- start start 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbf8102040 0x7fcbf81034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8102a00 0x7fcbf81039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810c9f0 con 0x7fcbf8100e60 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810c870 con 0x7fcbf8102a00 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810cb70 con 0x7fcbf8102040 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34924/0 (socket says 172.21.15.37:34924) 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 -- 172.21.15.37:0/406035006 learned_addr learned my addr 172.21.15.37:0/406035006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbf7fff640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbf8102040 0x7fcbf81034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfd132640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8102a00 0x7fcbf81039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 -- 172.21.15.37:0/406035006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbf8102040 msgr2=0x7fcbf81034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbf8102040 0x7fcbf81034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 -- 172.21.15.37:0/406035006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8102a00 msgr2=0x7fcbf81039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8102a00 0x7fcbf81039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 -- 172.21.15.37:0/406035006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf8104280 con 0x7fcbf8100e60 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfc931640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8104e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8006010 tx=0x7fcbe80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe803f030 con 0x7fcbf8100e60 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe80318f0 con 0x7fcbf8100e60 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf81a22d0 con 0x7fcbf8100e60 2024-04-23T18:17:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.539+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8035070 con 0x7fcbf8100e60 2024-04-23T18:17:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.540+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf81a2820 con 0x7fcbf8100e60 2024-04-23T18:17:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.541+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf81a2a80 con 0x7fcbf8100e60 2024-04-23T18:17:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.542+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcbe8042030 con 0x7fcbf8100e60 2024-04-23T18:17:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.542+0000 7fcbf5ffb640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcbe00a3bb0 0x7fcbe00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.542+0000 7fcbf7fff640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcbe00a3bb0 0x7fcbe00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.542+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcbe80f04c0 con 0x7fcbf8100e60 2024-04-23T18:17:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.543+0000 7fcbf7fff640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcbe00a3bb0 0x7fcbe00a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8104be0 tx=0x7fcbec00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.547+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbe80bd340 con 0x7fcbf8100e60 2024-04-23T18:17:57.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.732+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbf81a2c50 con 0x7fcbf8100e60 2024-04-23T18:17:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.732+0000 7fcbf5ffb640 1 -- 172.21.15.37:0/406035006 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fcbe80bcc40 con 0x7fcbf8100e60 2024-04-23T18:17:57.727 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcbe00a3bb0 msgr2=0x7fcbe00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcbe00a3bb0 0x7fcbe00a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8104be0 tx=0x7fcbec00a250 comp rx=0 tx=0).stop 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 msgr2=0x7fcbf8104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8104e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8006010 tx=0x7fcbe80042f0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 shutdown_connections 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcbe00a3bb0 0x7fcbe00a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8102a00 0x7fcbf81039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbf8102040 0x7fcbf81034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 --2- 172.21.15.37:0/406035006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf8100e60 0x7fcbf8104e70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.734+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 >> 172.21.15.37:0/406035006 conn(0x7fcbf80fc440 msgr2=0x7fcbf806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.735+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 shutdown_connections 2024-04-23T18:17:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.735+0000 7fcbfeb95640 1 -- 172.21.15.37:0/406035006 wait complete. 2024-04-23T18:17:57.740 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:17:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 -- 172.21.15.37:0/267403050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7301053a0 msgr2=0x7fa730107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 --2- 172.21.15.37:0/267403050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7301053a0 0x7fa730107790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa724009900 tx=0x7fa72402a410 comp rx=0 tx=0).stop 2024-04-23T18:17:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 -- 172.21.15.37:0/267403050 shutdown_connections 2024-04-23T18:17:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 --2- 172.21.15.37:0/267403050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa730107cd0 0x7fa73010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 --2- 172.21.15.37:0/267403050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7301053a0 0x7fa730107790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 --2- 172.21.15.37:0/267403050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 0x7fa730104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 -- 172.21.15.37:0/267403050 >> 172.21.15.37:0/267403050 conn(0x7fa7300fc400 msgr2=0x7fa7300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 -- 172.21.15.37:0/267403050 shutdown_connections 2024-04-23T18:17:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.847+0000 7fa73682c640 1 -- 172.21.15.37:0/267403050 wait complete. 2024-04-23T18:17:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 Processor -- start 2024-04-23T18:17:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 -- start start 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 0x7fa730198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7301053a0 0x7fa730198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 0x7fa73019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa73010db10 con 0x7fa730107cd0 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa73010d990 con 0x7fa730102a70 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa73682c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa73010dc90 con 0x7fa7301053a0 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.848+0000 7fa734dc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 0x7fa73019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 0x7fa73019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34942/0 (socket says 172.21.15.37:34942) 2024-04-23T18:17:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 -- 172.21.15.37:0/2566525058 learned_addr learned my addr 172.21.15.37:0/2566525058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72f7fe640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7301053a0 0x7fa730198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72ffff640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 0x7fa730198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 -- 172.21.15.37:0/2566525058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7301053a0 msgr2=0x7fa730198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7301053a0 0x7fa730198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 -- 172.21.15.37:0/2566525058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 msgr2=0x7fa730198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 0x7fa730198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 -- 172.21.15.37:0/2566525058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73019fe30 con 0x7fa730107cd0 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72ffff640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 0x7fa730198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72f7fe640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7301053a0 0x7fa730198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa734dc9640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 0x7fa73019f7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa72800eb10 tx=0x7fa72800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728004b20 con 0x7fa730107cd0 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7301a0120 con 0x7fa730107cd0 2024-04-23T18:17:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa728004cc0 con 0x7fa730107cd0 2024-04-23T18:17:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728010670 con 0x7fa730107cd0 2024-04-23T18:17:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.849+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7301a0640 con 0x7fa730107cd0 2024-04-23T18:17:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.850+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7300fdef0 con 0x7fa730107cd0 2024-04-23T18:17:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.851+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa728005000 con 0x7fa730107cd0 2024-04-23T18:17:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.852+0000 7fa72d7fa640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7180a3bb0 0x7fa7180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.852+0000 7fa72ffff640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7180a3bb0 0x7fa7180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.852+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa7280d0740 con 0x7fa730107cd0 2024-04-23T18:17:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.853+0000 7fa72ffff640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7180a3bb0 0x7fa7180a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7200058c0 tx=0x7fa720005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:57.856+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa728053ff0 con 0x7fa730107cd0 2024-04-23T18:17:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.081+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa73002d5c0 con 0x7fa730107cd0 2024-04-23T18:17:58.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.081+0000 7fa72d7fa640 1 -- 172.21.15.37:0/2566525058 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa72809d5c0 con 0x7fa730107cd0 2024-04-23T18:17:58.075 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:17:58.076 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-23T18:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 1 2024-04-23T18:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7180a3bb0 msgr2=0x7fa7180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7180a3bb0 0x7fa7180a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7200058c0 tx=0x7fa720005850 comp rx=0 tx=0).stop 2024-04-23T18:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 msgr2=0x7fa73019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 0x7fa73019f7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa72800eb10 tx=0x7fa72800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 shutdown_connections 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7180a3bb0 0x7fa7180a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa730107cd0 0x7fa73019f7c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7301053a0 0x7fa730198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 --2- 172.21.15.37:0/2566525058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa730102a70 0x7fa730198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 >> 172.21.15.37:0/2566525058 conn(0x7fa7300fc400 msgr2=0x7fa7301034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 shutdown_connections 2024-04-23T18:17:58.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:17:58.083+0000 7fa73682c640 1 -- 172.21.15.37:0/2566525058 wait complete. 2024-04-23T18:17:58.087 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3484045335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:17:58.087 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/406035006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:17:58.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3484045335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:17:58.282 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/406035006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:17:58.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3484045335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:17:58.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/406035006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:17:58.606 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 systemd[1]: Starting Ceph mds.l for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 podman[62566]: 2024-04-23 18:17:58.581323811 +0000 UTC m=+0.017892270 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 podman[62566]: 2024-04-23 18:17:58.691360566 +0000 UTC m=+0.127929064 container create ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 podman[62566]: 2024-04-23 18:17:58.893813237 +0000 UTC m=+0.330381698 container init ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 podman[62566]: 2024-04-23 18:17:58.89651247 +0000 UTC m=+0.333080926 container start ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-mds[62585]: -- 172.21.15.173:0/3074115857 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e209305a40 con 0x55e2092d8800 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l[62581]: starting mds.l at 2024-04-23T18:17:58.979 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 bash[62566]: ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 2024-04-23T18:17:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:58 smithi172 ceph-mon[32456]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:58 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2566525058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:17:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:58 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6832/2179419272,v1:172.21.15.173:6833/2179419272] up:boot 2024-04-23T18:17:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:58 smithi172 ceph-mon[32456]: fsmap 1 up:standby 2024-04-23T18:17:59.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:17:58 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:17:59.355 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:17:58 smithi173 systemd[1]: Started Ceph mds.l for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:17:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-mon[32906]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2566525058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:17:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6832/2179419272,v1:172.21.15.173:6833/2179419272] up:boot 2024-04-23T18:17:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-mon[32906]: fsmap 1 up:standby 2024-04-23T18:17:59.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:17:58 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:17:59.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:58 smithi037 ceph-mon[29404]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:17:59.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2566525058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:17:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:58 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6832/2179419272,v1:172.21.15.173:6833/2179419272] up:boot 2024-04-23T18:17:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:58 smithi037 ceph-mon[29404]: fsmap 1 up:standby 2024-04-23T18:17:59.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:17:58 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:18:00.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:00.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 -- 172.21.15.37:0/722829064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538101fe0 msgr2=0x7f0538102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:00.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 --2- 172.21.15.37:0/722829064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538101fe0 0x7f0538102460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f052c009900 tx=0x7f052c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 -- 172.21.15.37:0/722829064 shutdown_connections 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 --2- 172.21.15.37:0/722829064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05381029a0 0x7f05381092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 --2- 172.21.15.37:0/722829064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538101fe0 0x7f0538102460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 --2- 172.21.15.37:0/722829064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0538100e90 0x7f05381012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 -- 172.21.15.37:0/722829064 >> 172.21.15.37:0/722829064 conn(0x7f05380fc400 msgr2=0x7f05380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 -- 172.21.15.37:0/722829064 shutdown_connections 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 -- 172.21.15.37:0/722829064 wait complete. 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 Processor -- start 2024-04-23T18:18:00.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.209+0000 7f053e4cb640 1 -- start start 2024-04-23T18:18:00.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 0x7f0538198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:00.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0538101fe0 0x7f0538198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:00.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05381029a0 0x7f053819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:00.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f053810c9d0 con 0x7f0538100e90 2024-04-23T18:18:00.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f053810c850 con 0x7f0538101fe0 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f053810cb50 con 0x7f05381029a0 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 0x7f0538198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 0x7f0538198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34958/0 (socket says 172.21.15.37:34958) 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 -- 172.21.15.37:0/4284673110 learned_addr learned my addr 172.21.15.37:0/4284673110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f05377fe640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0538101fe0 0x7f0538198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053ca68640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05381029a0 0x7f053819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 -- 172.21.15.37:0/4284673110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05381029a0 msgr2=0x7f053819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05381029a0 0x7f053819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 -- 172.21.15.37:0/4284673110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0538101fe0 msgr2=0x7f0538198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0538101fe0 0x7f0538198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 -- 172.21.15.37:0/4284673110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053819fd20 con 0x7f0538100e90 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053ca68640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05381029a0 0x7f053819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:00.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f0537fff640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 0x7f0538198100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f052800c920 tx=0x7f052800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528007c70 con 0x7f0538100e90 2024-04-23T18:18:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0528007e10 con 0x7f0538100e90 2024-04-23T18:18:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053819ffb0 con 0x7f0538100e90 2024-04-23T18:18:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.210+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528018820 con 0x7f0538100e90 2024-04-23T18:18:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.211+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05381a04c0 con 0x7f0538100e90 2024-04-23T18:18:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.212+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05381a0750 con 0x7f0538100e90 2024-04-23T18:18:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.213+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f052800f450 con 0x7f0538100e90 2024-04-23T18:18:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.213+0000 7f05357fa640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f05200a3c30 0x7f05200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.214+0000 7f05377fe640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f05200a3c30 0x7f05200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.214+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f05280d02c0 con 0x7f0538100e90 2024-04-23T18:18:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.214+0000 7f05377fe640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f05200a3c30 0x7f05200a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f052c009230 tx=0x7f052c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:00.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.217+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0528014030 con 0x7f0538100e90 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:00.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:00 smithi172 ceph-mon[32456]: Deploying daemon mds.k on smithi172 2024-04-23T18:18:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:00.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:00.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:00 smithi173 ceph-mon[32906]: Deploying daemon mds.k on smithi172 2024-04-23T18:18:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:00.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:00.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:00.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:00.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:00 smithi037 ceph-mon[29404]: Deploying daemon mds.k on smithi172 2024-04-23T18:18:00.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.443+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05381a0a40 con 0x7f0538100e90 2024-04-23T18:18:00.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.444+0000 7f05357fa640 1 -- 172.21.15.37:0/4284673110 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f052809d250 con 0x7f0538100e90 2024-04-23T18:18:00.439 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:00.439 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:18:00.439 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-23T18:18:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f05200a3c30 msgr2=0x7f05200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f05200a3c30 0x7f05200a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f052c009230 tx=0x7f052c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 msgr2=0x7f0538198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 0x7f0538198100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f052800c920 tx=0x7f052800cde0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 shutdown_connections 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f05200a3c30 0x7f05200a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05381029a0 0x7f053819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0538101fe0 0x7f0538198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 --2- 172.21.15.37:0/4284673110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0538100e90 0x7f0538198100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 >> 172.21.15.37:0/4284673110 conn(0x7f05380fc400 msgr2=0x7f05380fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 shutdown_connections 2024-04-23T18:18:00.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:00.446+0000 7f053e4cb640 1 -- 172.21.15.37:0/4284673110 wait complete. 2024-04-23T18:18:01.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:01 smithi172 ceph-mon[32456]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:01.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:01 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4284673110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:01.338 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 systemd[1]: Starting Ceph mds.k for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:01.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:01 smithi173 ceph-mon[32906]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:01.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:01 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4284673110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:01.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:01 smithi037 ceph-mon[29404]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:01.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:01 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4284673110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:01.765 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 podman[61245]: 2024-04-23 18:18:01.267239922 +0000 UTC m=+0.018144973 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:01.765 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 podman[61245]: 2024-04-23 18:18:01.393514 +0000 UTC m=+0.144419039 container create ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-23T18:18:01.765 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 podman[61245]: 2024-04-23 18:18:01.771942374 +0000 UTC m=+0.522847415 container init ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:18:02.088 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 podman[61245]: 2024-04-23 18:18:01.77459683 +0000 UTC m=+0.525501877 container start ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T18:18:02.088 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 ceph-mds[61264]: -- 172.21.15.172:0/3551784396 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555918c31a40 con 0x555918c04c00 2024-04-23T18:18:02.088 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k[61260]: starting mds.k at 2024-04-23T18:18:02.088 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 bash[61245]: ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 2024-04-23T18:18:02.088 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:18:01 smithi172 systemd[1]: Started Ceph mds.k for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:02.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 -- 172.21.15.37:0/102536128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450103190 msgr2=0x7f7450103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 --2- 172.21.15.37:0/102536128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450103190 0x7f7450103610 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7444009900 tx=0x7f744402a410 comp rx=0 tx=0).stop 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 -- 172.21.15.37:0/102536128 shutdown_connections 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 --2- 172.21.15.37:0/102536128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7450103b50 0x7f745010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 --2- 172.21.15.37:0/102536128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450103190 0x7f7450103610 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 --2- 172.21.15.37:0/102536128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450101fa0 0x7f74501023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.564+0000 7f7456260640 1 -- 172.21.15.37:0/102536128 >> 172.21.15.37:0/102536128 conn(0x7f74500fd530 msgr2=0x7f74500ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 -- 172.21.15.37:0/102536128 shutdown_connections 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 -- 172.21.15.37:0/102536128 wait complete. 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 Processor -- start 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 -- start start 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 0x7f7450198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7450103190 0x7f74501987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450103b50 0x7f745019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f745010db40 con 0x7f7450101fa0 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f745010d9c0 con 0x7f7450103190 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f7456260640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f745010dcc0 con 0x7f7450103b50 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.565+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 0x7f7450198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:02.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 0x7f7450198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34976/0 (socket says 172.21.15.37:34976) 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 -- 172.21.15.37:0/1835229733 learned_addr learned my addr 172.21.15.37:0/1835229733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744effd640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7450103190 0x7f74501987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744ffff640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450103b50 0x7f745019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 -- 172.21.15.37:0/1835229733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450103b50 msgr2=0x7f745019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450103b50 0x7f745019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 -- 172.21.15.37:0/1835229733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7450103190 msgr2=0x7f74501987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7450103190 0x7f74501987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 -- 172.21.15.37:0/1835229733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745019feb0 con 0x7f7450101fa0 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744ffff640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450103b50 0x7f745019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744f7fe640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 0x7f7450198290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f744000b450 tx=0x7f744000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440015020 con 0x7f7450101fa0 2024-04-23T18:18:02.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74501a01a0 con 0x7f7450101fa0 2024-04-23T18:18:02.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7440011d40 con 0x7f7450101fa0 2024-04-23T18:18:02.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744000f340 con 0x7f7450101fa0 2024-04-23T18:18:02.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.566+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a0740 con 0x7f7450101fa0 2024-04-23T18:18:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.567+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7450109390 con 0x7f7450101fa0 2024-04-23T18:18:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.568+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7440018020 con 0x7f7450101fa0 2024-04-23T18:18:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.569+0000 7f744cff9640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f74380a3bb0 0x7f74380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.569+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f74400cfa00 con 0x7f7450101fa0 2024-04-23T18:18:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.569+0000 7f744effd640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f74380a3bb0 0x7f74380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.569+0000 7f744effd640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f74380a3bb0 0x7f74380a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7444000c00 tx=0x7f74440314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.574+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f744001f020 con 0x7f7450101fa0 2024-04-23T18:18:02.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.805+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f745002d5c0 con 0x7f7450101fa0 2024-04-23T18:18:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.806+0000 7f744cff9640 1 -- 172.21.15.37:0/1835229733 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f744000d040 con 0x7f7450101fa0 2024-04-23T18:18:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:02.800 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:18:02.801 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-23T18:18:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f74380a3bb0 msgr2=0x7f74380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:02.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f74380a3bb0 0x7f74380a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7444000c00 tx=0x7f74440314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 msgr2=0x7f7450198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 0x7f7450198290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f744000b450 tx=0x7f744000b910 comp rx=0 tx=0).stop 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 shutdown_connections 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f74380a3bb0 0x7f74380a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7450103b50 0x7f745019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7450103190 0x7f74501987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 --2- 172.21.15.37:0/1835229733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7450101fa0 0x7f7450198290 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.808+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 >> 172.21.15.37:0/1835229733 conn(0x7f74500fd530 msgr2=0x7f7450106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.809+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 shutdown_connections 2024-04-23T18:18:02.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:02.809+0000 7f7456260640 1 -- 172.21.15.37:0/1835229733 wait complete. 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: Deploying daemon mds.j on smithi037 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6832/3950527293,v1:172.21.15.172:6833/3950527293] up:boot 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: fsmap 2 up:standby 2024-04-23T18:18:03.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:18:03.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:02 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1835229733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: Deploying daemon mds.j on smithi037 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6832/3950527293,v1:172.21.15.172:6833/3950527293] up:boot 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: fsmap 2 up:standby 2024-04-23T18:18:03.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:18:03.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:02 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1835229733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:03.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:03.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: Deploying daemon mds.j on smithi037 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6832/3950527293,v1:172.21.15.172:6833/3950527293] up:boot 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: fsmap 2 up:standby 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:18:03.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:02 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1835229733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:04.064 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:03 smithi037 systemd[1]: Starting Ceph mds.j for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:04.371 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 podman[67288]: 2024-04-23 18:18:04.198493717 +0000 UTC m=+0.104915748 container create f7505dc57dfa70fa8751643e58e0e69694e9194d6a6286e4e5a0ef52778ce8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:18:04.371 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 podman[67288]: 2024-04-23 18:18:04.112163766 +0000 UTC m=+0.018585756 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:04.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:04.839 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 podman[67288]: 2024-04-23 18:18:04.401275932 +0000 UTC m=+0.307697921 container init f7505dc57dfa70fa8751643e58e0e69694e9194d6a6286e4e5a0ef52778ce8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T18:18:04.839 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 podman[67288]: 2024-04-23 18:18:04.403968963 +0000 UTC m=+0.310390959 container start f7505dc57dfa70fa8751643e58e0e69694e9194d6a6286e4e5a0ef52778ce8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:18:04.839 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 ceph-mds[67307]: -- 172.21.15.37:0/225248872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55673dc4fa40 con 0x55673dc22c00 2024-04-23T18:18:04.839 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j[67303]: starting mds.j at 2024-04-23T18:18:04.839 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 bash[67288]: f7505dc57dfa70fa8751643e58e0e69694e9194d6a6286e4e5a0ef52778ce8ff 2024-04-23T18:18:04.839 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:18:04 smithi037 systemd[1]: Started Ceph mds.j for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 -- 172.21.15.37:0/3422563258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184102a10 msgr2=0x7f1184104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 --2- 172.21.15.37:0/3422563258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184102a10 0x7f1184104e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002a410 comp rx=0 tx=0).stop 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 -- 172.21.15.37:0/3422563258 shutdown_connections 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 --2- 172.21.15.37:0/3422563258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184107c70 0x7f118410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 --2- 172.21.15.37:0/3422563258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1184105340 0x7f1184107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 --2- 172.21.15.37:0/3422563258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184102a10 0x7f1184104e00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 -- 172.21.15.37:0/3422563258 >> 172.21.15.37:0/3422563258 conn(0x7f11840fc400 msgr2=0x7f11840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 -- 172.21.15.37:0/3422563258 shutdown_connections 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.929+0000 7f118bcbb640 1 -- 172.21.15.37:0/3422563258 wait complete. 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 Processor -- start 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 -- start start 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184102a10 0x7f1184198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 0x7f1184198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1184107c70 0x7f118419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f118410dae0 con 0x7f1184105340 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f118410d960 con 0x7f1184107c70 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118bcbb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f118410dc60 con 0x7f1184102a10 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f1189256640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 0x7f1184198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f1189256640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 0x7f1184198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39750/0 (socket says 172.21.15.37:39750) 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f1189256640 1 -- 172.21.15.37:0/2682486371 learned_addr learned my addr 172.21.15.37:0/2682486371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f118a258640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1184107c70 0x7f118419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.930+0000 7f1189a57640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184102a10 0x7f1184198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189256640 1 -- 172.21.15.37:0/2682486371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184102a10 msgr2=0x7f1184198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189256640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184102a10 0x7f1184198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189256640 1 -- 172.21.15.37:0/2682486371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1184107c70 msgr2=0x7f118419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189256640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1184107c70 0x7f118419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189256640 1 -- 172.21.15.37:0/2682486371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118419fe70 con 0x7f1184105340 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189a57640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184102a10 0x7f1184198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f1189256640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 0x7f1184198790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f117400e9c0 tx=0x7f117400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117400cd80 con 0x7f1184105340 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11841a0160 con 0x7f1184105340 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11740054d0 con 0x7f1184105340 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174010430 con 0x7f1184105340 2024-04-23T18:18:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.931+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11841a0700 con 0x7f1184105340 2024-04-23T18:18:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.932+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11840fdef0 con 0x7f1184105340 2024-04-23T18:18:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.933+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1174004fd0 con 0x7f1184105340 2024-04-23T18:18:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.934+0000 7f117affd640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f11700a3bb0 0x7f11700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.934+0000 7f1189a57640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f11700a3bb0 0x7f11700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.934+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f11740d0400 con 0x7f1184105340 2024-04-23T18:18:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.934+0000 7f1189a57640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f11700a3bb0 0x7f11700a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f11800314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:04.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:04.938+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f117409d280 con 0x7f1184105340 2024-04-23T18:18:05.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.178+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f118402d770 con 0x7f1184105340 2024-04-23T18:18:05.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.179+0000 7f117affd640 1 -- 172.21.15.37:0/2682486371 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f117409cb80 con 0x7f1184105340 2024-04-23T18:18:05.173 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:05.173 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:18:05.173 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-04-23T18:18:05.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f11700a3bb0 msgr2=0x7f11700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:05.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f11700a3bb0 0x7f11700a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f11800314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:05.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 msgr2=0x7f1184198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:05.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 0x7f1184198790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f117400e9c0 tx=0x7f117400ee80 comp rx=0 tx=0).stop 2024-04-23T18:18:05.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 shutdown_connections 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f11700a3bb0 0x7f11700a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1184107c70 0x7f118419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1184105340 0x7f1184198790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 --2- 172.21.15.37:0/2682486371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1184102a10 0x7f1184198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 >> 172.21.15.37:0/2682486371 conn(0x7f11840fc400 msgr2=0x7f1184103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 shutdown_connections 2024-04-23T18:18:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:05.181+0000 7f118bcbb640 1 -- 172.21.15.37:0/2682486371 wait complete. 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: Deploying daemon mds.d on smithi037 2024-04-23T18:18:05.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.37:6834/3749554731,v1:172.21.15.37:6835/3749554731] up:boot 2024-04-23T18:18:05.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: fsmap 3 up:standby 2024-04-23T18:18:05.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:18:05.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:05 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2682486371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:05.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:05.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: Deploying daemon mds.d on smithi037 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.37:6834/3749554731,v1:172.21.15.37:6835/3749554731] up:boot 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: fsmap 3 up:standby 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:18:05.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:05 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2682486371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: Deploying daemon mds.d on smithi037 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.37:6834/3749554731,v1:172.21.15.37:6835/3749554731] up:boot 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: fsmap 3 up:standby 2024-04-23T18:18:05.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:18:05.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:05 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2682486371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:06.871 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:06 smithi037 systemd[1]: Starting Ceph mds.d for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:07.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:07.209 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:06 smithi037 podman[67640]: 2024-04-23 18:18:06.85613284 +0000 UTC m=+0.018697121 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:07.209 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:06 smithi037 podman[67640]: 2024-04-23 18:18:06.970689457 +0000 UTC m=+0.133253766 container create 3fb45d89b5314b6188429aa39df58260b1ebd088250c3ab6f7268c2f3cf45a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:18:07.209 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:07 smithi037 podman[67640]: 2024-04-23 18:18:07.164501513 +0000 UTC m=+0.327065781 container init 3fb45d89b5314b6188429aa39df58260b1ebd088250c3ab6f7268c2f3cf45a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:18:07.209 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:07 smithi037 podman[67640]: 2024-04-23 18:18:07.167144748 +0000 UTC m=+0.329709024 container start 3fb45d89b5314b6188429aa39df58260b1ebd088250c3ab6f7268c2f3cf45a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:18:07.210 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mds[67659]: -- 172.21.15.37:0/710206703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a56503ba40 con 0x55a56500f000 2024-04-23T18:18:07.210 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d[67655]: starting mds.d at 2024-04-23T18:18:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 -- 172.21.15.37:0/3244471124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c1029b0 msgr2=0x7fef6c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 --2- 172.21.15.37:0/3244471124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c1029b0 0x7fef6c1092d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef64009be0 tx=0x7fef6402a6c0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 -- 172.21.15.37:0/3244471124 shutdown_connections 2024-04-23T18:18:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 --2- 172.21.15.37:0/3244471124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c1029b0 0x7fef6c1092d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 --2- 172.21.15.37:0/3244471124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef6c101ff0 0x7fef6c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 --2- 172.21.15.37:0/3244471124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c100ea0 0x7fef6c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 -- 172.21.15.37:0/3244471124 >> 172.21.15.37:0/3244471124 conn(0x7fef6c0fc430 msgr2=0x7fef6c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 -- 172.21.15.37:0/3244471124 shutdown_connections 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 -- 172.21.15.37:0/3244471124 wait complete. 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 Processor -- start 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.304+0000 7fef73813640 1 -- start start 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef73813640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef6c100ea0 0x7fef6c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef73813640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c101ff0 0x7fef6c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef73813640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 0x7fef6c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef73813640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10c9f0 con 0x7fef6c1029b0 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef73813640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10c870 con 0x7fef6c100ea0 2024-04-23T18:18:07.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef73813640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10cb70 con 0x7fef6c101ff0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 0x7fef6c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 0x7fef6c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39794/0 (socket says 172.21.15.37:39794) 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 -- 172.21.15.37:0/578452965 learned_addr learned my addr 172.21.15.37:0/578452965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef715af640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef6c100ea0 0x7fef6c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef70dae640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c101ff0 0x7fef6c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 -- 172.21.15.37:0/578452965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c101ff0 msgr2=0x7fef6c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c101ff0 0x7fef6c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 -- 172.21.15.37:0/578452965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef6c100ea0 msgr2=0x7fef6c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef6c100ea0 0x7fef6c06b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 -- 172.21.15.37:0/578452965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c19fdf0 con 0x7fef6c1029b0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.305+0000 7fef71db0640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 0x7fef6c19f4e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef64034040 tx=0x7fef64030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.308+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64041020 con 0x7fef6c1029b0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.308+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c1a0020 con 0x7fef6c1029b0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.308+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef640318f0 con 0x7fef6c1029b0 2024-04-23T18:18:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.308+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a0560 con 0x7fef6c1029b0 2024-04-23T18:18:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.308+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64036040 con 0x7fef6c1029b0 2024-04-23T18:18:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.309+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c0691c0 con 0x7fef6c1029b0 2024-04-23T18:18:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.310+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef64044020 con 0x7fef6c1029b0 2024-04-23T18:18:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.311+0000 7fef627fc640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef580a3bb0 0x7fef580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.311+0000 7fef715af640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef580a3bb0 0x7fef580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.311+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fef640f0dc0 con 0x7fef6c1029b0 2024-04-23T18:18:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.311+0000 7fef715af640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef580a3bb0 0x7fef580a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005fd0 tx=0x7fef5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:07.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.315+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef6404b020 con 0x7fef6c1029b0 2024-04-23T18:18:07.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.540+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef6c02d5f0 con 0x7fef6c1029b0 2024-04-23T18:18:07.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.540+0000 7fef627fc640 1 -- 172.21.15.37:0/578452965 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fef640bdc40 con 0x7fef6c1029b0 2024-04-23T18:18:07.535 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:07.535 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:18:07.535 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef580a3bb0 msgr2=0x7fef580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef580a3bb0 0x7fef580a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef5c005fd0 tx=0x7fef5c009260 comp rx=0 tx=0).stop 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 msgr2=0x7fef6c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 0x7fef6c19f4e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef64034040 tx=0x7fef64030d70 comp rx=0 tx=0).stop 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 shutdown_connections 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef580a3bb0 0x7fef580a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef6c1029b0 0x7fef6c19f4e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef6c101ff0 0x7fef6c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 --2- 172.21.15.37:0/578452965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef6c100ea0 0x7fef6c06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 >> 172.21.15.37:0/578452965 conn(0x7fef6c0fc430 msgr2=0x7fef6c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 shutdown_connections 2024-04-23T18:18:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:07.542+0000 7fef73813640 1 -- 172.21.15.37:0/578452965 wait complete. 2024-04-23T18:18:07.547 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:07 smithi037 bash[67640]: 3fb45d89b5314b6188429aa39df58260b1ebd088250c3ab6f7268c2f3cf45a0b 2024-04-23T18:18:07.547 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:18:07 smithi037 systemd[1]: Started Ceph mds.d for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:07.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:07.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:07 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/578452965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:07.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:07.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:07.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:07 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/578452965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:07.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:07.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:07 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/578452965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:08 smithi172 ceph-mon[32456]: Deploying daemon mds.e on smithi172 2024-04-23T18:18:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:08 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.37:6836/3961977132,v1:172.21.15.37:6837/3961977132] up:boot 2024-04-23T18:18:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:08 smithi172 ceph-mon[32456]: fsmap 4 up:standby 2024-04-23T18:18:08.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:08 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:18:08.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:08 smithi173 ceph-mon[32906]: Deploying daemon mds.e on smithi172 2024-04-23T18:18:08.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:08 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.37:6836/3961977132,v1:172.21.15.37:6837/3961977132] up:boot 2024-04-23T18:18:08.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:08 smithi173 ceph-mon[32906]: fsmap 4 up:standby 2024-04-23T18:18:08.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:08 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:18:08.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:08 smithi037 ceph-mon[29404]: Deploying daemon mds.e on smithi172 2024-04-23T18:18:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:08 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.37:6836/3961977132,v1:172.21.15.37:6837/3961977132] up:boot 2024-04-23T18:18:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:08 smithi037 ceph-mon[29404]: fsmap 4 up:standby 2024-04-23T18:18:08.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:08 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:18:09.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:09.555 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 systemd[1]: Starting Ceph mds.e for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 -- 172.21.15.37:0/3029070320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f41031b0 msgr2=0x7f88f4103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3029070320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f41031b0 0x7f88f4103630 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 -- 172.21.15.37:0/3029070320 shutdown_connections 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3029070320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f4103b70 0x7f88f410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3029070320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f41031b0 0x7f88f4103630 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3029070320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f4101fc0 0x7f88f41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 -- 172.21.15.37:0/3029070320 >> 172.21.15.37:0/3029070320 conn(0x7f88f40fd550 msgr2=0x7f88f40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 -- 172.21.15.37:0/3029070320 shutdown_connections 2024-04-23T18:18:09.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.670+0000 7f88f8a20640 1 -- 172.21.15.37:0/3029070320 wait complete. 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 Processor -- start 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 -- start start 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 0x7f88f4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f41031b0 0x7f88f4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f4103b70 0x7f88f4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88f410db00 con 0x7f88f4101fc0 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f88f410d980 con 0x7f88f41031b0 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.671+0000 7f88f8a20640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88f410dc80 con 0x7f88f4103b70 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 0x7f88f4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 0x7f88f4075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39804/0 (socket says 172.21.15.37:39804) 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 -- 172.21.15.37:0/3401188382 learned_addr learned my addr 172.21.15.37:0/3401188382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:09.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f2d9d640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f4103b70 0x7f88f4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88ebfff640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f41031b0 0x7f88f4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 -- 172.21.15.37:0/3401188382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f4103b70 msgr2=0x7f88f4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f4103b70 0x7f88f4074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 -- 172.21.15.37:0/3401188382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f41031b0 msgr2=0x7f88f4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f41031b0 0x7f88f4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 -- 172.21.15.37:0/3401188382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f4075110 con 0x7f88f4101fc0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f259c640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 0x7f88f4075d00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f88e400c970 tx=0x7f88e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4007c10 con 0x7f88f4101fc0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f41a2280 con 0x7f88f4101fc0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e4007db0 con 0x7f88f4101fc0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e400f7d0 con 0x7f88f4101fc0 2024-04-23T18:18:09.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.672+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41a27f0 con 0x7f88f4101fc0 2024-04-23T18:18:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.674+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f41092e0 con 0x7f88f4101fc0 2024-04-23T18:18:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.675+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88e4018480 con 0x7f88f4101fc0 2024-04-23T18:18:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.675+0000 7f88eb7fe640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f88e00a3bb0 0x7f88e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.675+0000 7f88ebfff640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f88e00a3bb0 0x7f88e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:09.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.675+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f88e40d0210 con 0x7f88f4101fc0 2024-04-23T18:18:09.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.676+0000 7f88ebfff640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f88e00a3bb0 0x7f88e00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f88dc000c00 tx=0x7f88dc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:09.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.680+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88e409d120 con 0x7f88f4101fc0 2024-04-23T18:18:09.825 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:09 smithi172 ceph-mon[32456]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:09.825 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 podman[61567]: 2024-04-23 18:18:09.562341774 +0000 UTC m=+0.107913434 container create f9baf81f3881ee8b1394373b6a3d49d5dd040eb72bb11034d57b3a90b603c1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T18:18:09.825 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 podman[61567]: 2024-04-23 18:18:09.473402561 +0000 UTC m=+0.018974220 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:09.825 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 podman[61567]: 2024-04-23 18:18:09.806737207 +0000 UTC m=+0.352308873 container init f9baf81f3881ee8b1394373b6a3d49d5dd040eb72bb11034d57b3a90b603c1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:18:09.825 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 podman[61567]: 2024-04-23 18:18:09.80926668 +0000 UTC m=+0.354838343 container start f9baf81f3881ee8b1394373b6a3d49d5dd040eb72bb11034d57b3a90b603c1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T18:18:09.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:09 smithi173 ceph-mon[32906]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:09.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:09 smithi037 ceph-mon[29404]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:09.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.906+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f402d5c0 con 0x7f88f4101fc0 2024-04-23T18:18:09.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.907+0000 7f88eb7fe640 1 -- 172.21.15.37:0/3401188382 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f88e409ca20 con 0x7f88f4101fc0 2024-04-23T18:18:09.902 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:09.902 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:18:09.902 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f88e00a3bb0 msgr2=0x7f88e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f88e00a3bb0 0x7f88e00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f88dc000c00 tx=0x7f88dc0314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 msgr2=0x7f88f4075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 0x7f88f4075d00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f88e400c970 tx=0x7f88e400ce30 comp rx=0 tx=0).stop 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 shutdown_connections 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f88e00a3bb0 0x7f88e00a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f4103b70 0x7f88f4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f41031b0 0x7f88f4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 --2- 172.21.15.37:0/3401188382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4101fc0 0x7f88f4075d00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:09.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 >> 172.21.15.37:0/3401188382 conn(0x7f88f40fd550 msgr2=0x7f88f41064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:09.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 shutdown_connections 2024-04-23T18:18:09.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:09.909+0000 7f88f8a20640 1 -- 172.21.15.37:0/3401188382 wait complete. 2024-04-23T18:18:10.087 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 ceph-mds[61587]: -- 172.21.15.172:0/193859599 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc36223a40 con 0x55fc361f7000 2024-04-23T18:18:10.088 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e[61583]: starting mds.e at 2024-04-23T18:18:10.088 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 bash[61567]: f9baf81f3881ee8b1394373b6a3d49d5dd040eb72bb11034d57b3a90b603c1b6 2024-04-23T18:18:10.088 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:18:09 smithi172 systemd[1]: Started Ceph mds.e for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3401188382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:10.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:10 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3401188382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:10.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:10 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:10.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3401188382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:10.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:10.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:10 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:11.830 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:11 smithi173 ceph-mon[32906]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:11.830 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:11 smithi173 ceph-mon[32906]: Deploying daemon mds.f on smithi173 2024-04-23T18:18:11.830 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:11 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6834/2640036963,v1:172.21.15.172:6835/2640036963] up:boot 2024-04-23T18:18:11.830 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:11 smithi173 ceph-mon[32906]: fsmap 5 up:standby 2024-04-23T18:18:11.830 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:11 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:18:11.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:11 smithi172 ceph-mon[32456]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:11 smithi172 ceph-mon[32456]: Deploying daemon mds.f on smithi173 2024-04-23T18:18:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:11 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6834/2640036963,v1:172.21.15.172:6835/2640036963] up:boot 2024-04-23T18:18:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:11 smithi172 ceph-mon[32456]: fsmap 5 up:standby 2024-04-23T18:18:11.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:11 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:18:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:11 smithi037 ceph-mon[29404]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:11 smithi037 ceph-mon[29404]: Deploying daemon mds.f on smithi173 2024-04-23T18:18:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:11 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6834/2640036963,v1:172.21.15.172:6835/2640036963] up:boot 2024-04-23T18:18:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:11 smithi037 ceph-mon[29404]: fsmap 5 up:standby 2024-04-23T18:18:11.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:11 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:18:11.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 -- 172.21.15.37:0/2595411810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc101fc0 msgr2=0x7f2cbc1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/2595411810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc101fc0 0x7f2cbc1023e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009900 tx=0x7f2cac02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 -- 172.21.15.37:0/2595411810 shutdown_connections 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/2595411810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cbc103b70 0x7f2cbc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/2595411810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cbc1031b0 0x7f2cbc103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/2595411810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc101fc0 0x7f2cbc1023e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 -- 172.21.15.37:0/2595411810 >> 172.21.15.37:0/2595411810 conn(0x7f2cbc0fd550 msgr2=0x7f2cbc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 -- 172.21.15.37:0/2595411810 shutdown_connections 2024-04-23T18:18:12.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.036+0000 7f2cc0a13640 1 -- 172.21.15.37:0/2595411810 wait complete. 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 Processor -- start 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 -- start start 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cbc101fc0 0x7f2cbc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cbc1031b0 0x7f2cbc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 0x7f2cbc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10db00 con 0x7f2cbc103b70 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d980 con 0x7f2cbc1031b0 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cc0a13640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10dc80 con 0x7f2cbc101fc0 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.037+0000 7f2cbad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 0x7f2cbc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 0x7f2cbc074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39820/0 (socket says 172.21.15.37:39820) 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 -- 172.21.15.37:0/837974817 learned_addr learned my addr 172.21.15.37:0/837974817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:12.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cba59c640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cbc101fc0 0x7f2cbc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cb9d9b640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cbc1031b0 0x7f2cbc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 -- 172.21.15.37:0/837974817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cbc101fc0 msgr2=0x7f2cbc075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cbc101fc0 0x7f2cbc075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 -- 172.21.15.37:0/837974817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cbc1031b0 msgr2=0x7f2cbc074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cbc1031b0 0x7f2cbc074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 -- 172.21.15.37:0/837974817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc075110 con 0x7f2cbc103b70 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cbad9d640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 0x7f2cbc074880 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400eb10 tx=0x7f2cb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4004b20 con 0x7f2cbc103b70 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cb4004cc0 con 0x7f2cbc103b70 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a2350 con 0x7f2cbc103b70 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4010670 con 0x7f2cbc103b70 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.038+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a2850 con 0x7f2cbc103b70 2024-04-23T18:18:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.039+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc1093a0 con 0x7f2cbc103b70 2024-04-23T18:18:12.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.040+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2cb4005020 con 0x7f2cbc103b70 2024-04-23T18:18:12.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.041+0000 7f2cab7fe640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:12.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.041+0000 7f2cba59c640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:12.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.041+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f2cb40d0600 con 0x7f2cbc103b70 2024-04-23T18:18:12.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.041+0000 7f2cba59c640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009230 tx=0x7f2cac0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:12.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.046+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cb409d480 con 0x7f2cbc103b70 2024-04-23T18:18:12.105 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 systemd[1]: Starting Ceph mds.f for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:12.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.273+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cbc062b50 con 0x7f2cbc103b70 2024-04-23T18:18:12.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.274+0000 7f2cab7fe640 1 -- 172.21.15.37:0/837974817 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f2cb409cd80 con 0x7f2cbc103b70 2024-04-23T18:18:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:12.269 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:18:12.269 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2ca40a3bb0 msgr2=0x7f2ca40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009230 tx=0x7f2cac0314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 msgr2=0x7f2cbc074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 0x7f2cbc074880 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400eb10 tx=0x7f2cb400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 shutdown_connections 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2ca40a3bb0 0x7f2ca40a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cbc103b70 0x7f2cbc074880 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cbc1031b0 0x7f2cbc074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 --2- 172.21.15.37:0/837974817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cbc101fc0 0x7f2cbc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 >> 172.21.15.37:0/837974817 conn(0x7f2cbc0fd550 msgr2=0x7f2cbc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:12.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 shutdown_connections 2024-04-23T18:18:12.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:12.276+0000 7f2cc0a13640 1 -- 172.21.15.37:0/837974817 wait complete. 2024-04-23T18:18:12.489 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 podman[62872]: 2024-04-23 18:18:12.133812508 +0000 UTC m=+0.018712467 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:12.489 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 podman[62872]: 2024-04-23 18:18:12.272275028 +0000 UTC m=+0.157174978 container create d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T18:18:12.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:12 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/837974817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:12.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:12 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/837974817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:12.855 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 podman[62872]: 2024-04-23 18:18:12.496471626 +0000 UTC m=+0.381371576 container init d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:18:12.855 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 podman[62872]: 2024-04-23 18:18:12.499040586 +0000 UTC m=+0.383940544 container start d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T18:18:12.855 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 ceph-mds[62890]: -- 172.21.15.173:0/3536348760 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590ba44fa40 con 0x5590ba422c00 2024-04-23T18:18:12.856 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f[62886]: starting mds.f at 2024-04-23T18:18:12.856 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 bash[62872]: d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 2024-04-23T18:18:12.856 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:18:12 smithi173 systemd[1]: Started Ceph mds.f for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:12.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:12 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/837974817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6834/2419231317,v1:172.21.15.173:6835/2419231317] up:boot 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: fsmap 6 up:standby 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:13.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:13 smithi172 ceph-mon[32456]: Deploying daemon mds.c on smithi173 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6834/2419231317,v1:172.21.15.173:6835/2419231317] up:boot 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: fsmap 6 up:standby 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:13.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:13 smithi173 ceph-mon[32906]: Deploying daemon mds.c on smithi173 2024-04-23T18:18:13.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6834/2419231317,v1:172.21.15.173:6835/2419231317] up:boot 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: fsmap 6 up:standby 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:13.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:13 smithi037 ceph-mon[29404]: Deploying daemon mds.c on smithi173 2024-04-23T18:18:14.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/82139667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 msgr2=0x7fa7bc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/82139667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc101280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009900 tx=0x7fa7b802a410 comp rx=0 tx=0).stop 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/82139667 shutdown_connections 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/82139667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7bc102a00 0x7fa7bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/82139667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102040 0x7fa7bc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/82139667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc101280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/82139667 >> 172.21.15.37:0/82139667 conn(0x7fa7bc0fc440 msgr2=0x7fa7bc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/82139667 shutdown_connections 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.397+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/82139667 wait complete. 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 Processor -- start 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 -- start start 2024-04-23T18:18:14.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7bc102040 0x7fa7bc197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102a00 0x7fa7bc19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc10c9f0 con 0x7fa7bc100e60 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc10c870 con 0x7fa7bc102a00 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c3ba1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc10cb70 con 0x7fa7bc102040 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39840/0 (socket says 172.21.15.37:39840) 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 -- 172.21.15.37:0/2227621510 learned_addr learned my addr 172.21.15.37:0/2227621510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c113c640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7bc102040 0x7fa7bc197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c213e640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102a00 0x7fa7bc19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 -- 172.21.15.37:0/2227621510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7bc102040 msgr2=0x7fa7bc197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7bc102040 0x7fa7bc197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 -- 172.21.15.37:0/2227621510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102a00 msgr2=0x7fa7bc19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102a00 0x7fa7bc19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 -- 172.21.15.37:0/2227621510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc19ed20 con 0x7fa7bc100e60 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c213e640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102a00 0x7fa7bc19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.398+0000 7fa7c193d640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc197100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8006010 tx=0x7fa7b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.399+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b803f030 con 0x7fa7bc100e60 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.399+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b8031910 con 0x7fa7bc100e60 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.399+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc19efb0 con 0x7fa7bc100e60 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.399+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8035070 con 0x7fa7bc100e60 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.399+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc19f410 con 0x7fa7bc100e60 2024-04-23T18:18:14.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.400+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7bc1082f0 con 0x7fa7bc100e60 2024-04-23T18:18:14.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.401+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7b8042030 con 0x7fa7bc100e60 2024-04-23T18:18:14.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.401+0000 7fa7b2ffd640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7a80a3c30 0x7fa7a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.401+0000 7fa7c113c640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7a80a3c30 0x7fa7a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.401+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa7b80f0450 con 0x7fa7bc100e60 2024-04-23T18:18:14.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.402+0000 7fa7c113c640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7a80a3c30 0x7fa7a80a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc198610 tx=0x7fa7ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:14.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.406+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7b80bd2d0 con 0x7fa7bc100e60 2024-04-23T18:18:14.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.632+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7bc02d810 con 0x7fa7bc100e60 2024-04-23T18:18:14.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.632+0000 7fa7b2ffd640 1 -- 172.21.15.37:0/2227621510 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fa7b80bcbd0 con 0x7fa7bc100e60 2024-04-23T18:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:14.626 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-23T18:18:14.626 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7a80a3c30 msgr2=0x7fa7a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7a80a3c30 0x7fa7a80a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc198610 tx=0x7fa7ac005f90 comp rx=0 tx=0).stop 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 msgr2=0x7fa7bc197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc197100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8006010 tx=0x7fa7b80042f0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 shutdown_connections 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7a80a3c30 0x7fa7a80a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc102a00 0x7fa7bc19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7bc102040 0x7fa7bc197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 --2- 172.21.15.37:0/2227621510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7bc100e60 0x7fa7bc197100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 >> 172.21.15.37:0/2227621510 conn(0x7fa7bc0fc440 msgr2=0x7fa7bc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 shutdown_connections 2024-04-23T18:18:14.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:14.634+0000 7fa7c3ba1640 1 -- 172.21.15.37:0/2227621510 wait complete. 2024-04-23T18:18:15.356 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:18:15 smithi173 systemd[1]: Started Ceph mds.c for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2227621510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:15.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:15 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2227621510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:15.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:15 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2227621510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:15.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:15 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:16.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1157945992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac100800 msgr2=0x7fdbac100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1157945992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac100800 0x7fdbac100c20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009ca0 tx=0x7fdb9c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1157945992 shutdown_connections 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1157945992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac101b00 0x7fdbac101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1157945992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbac101160 0x7fdbac1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1157945992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac100800 0x7fdbac100c20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1157945992 >> 172.21.15.37:0/1157945992 conn(0x7fdbac0fc400 msgr2=0x7fdbac0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1157945992 shutdown_connections 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.760+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1157945992 wait complete. 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 Processor -- start 2024-04-23T18:18:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 -- start start 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac100800 0x7fdbac19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 0x7fdbac19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbac101b00 0x7fdbac1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac10fcb0 con 0x7fdbac101160 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac10fb30 con 0x7fdbac100800 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb39ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac10fe30 con 0x7fdbac101b00 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb0f8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 0x7fdbac19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb0f8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 0x7fdbac19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36822/0 (socket says 172.21.15.37:36822) 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb0f8a640 1 -- 172.21.15.37:0/1222260665 learned_addr learned my addr 172.21.15.37:0/1222260665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb178b640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac100800 0x7fdbac19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.761+0000 7fdbb1f8c640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbac101b00 0x7fdbac1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb0f8a640 1 -- 172.21.15.37:0/1222260665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbac101b00 msgr2=0x7fdbac1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb0f8a640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbac101b00 0x7fdbac1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb0f8a640 1 -- 172.21.15.37:0/1222260665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac100800 msgr2=0x7fdbac19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:16.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb0f8a640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac100800 0x7fdbac19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb0f8a640 1 -- 172.21.15.37:0/1222260665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac1a2000 con 0x7fdbac101160 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb178b640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac100800 0x7fdbac19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb0f8a640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 0x7fdbac19a970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d950 tx=0x7fdba000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0017070 con 0x7fdbac101160 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac1a22f0 con 0x7fdbac101160 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba0009c30 con 0x7fdbac101160 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba000f620 con 0x7fdbac101160 2024-04-23T18:18:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.762+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac1a2840 con 0x7fdbac101160 2024-04-23T18:18:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.763+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbac10b7d0 con 0x7fdbac101160 2024-04-23T18:18:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.764+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdba0010460 con 0x7fdbac101160 2024-04-23T18:18:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.765+0000 7fdbaa7fc640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb940a3bb0 0x7fdb940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.765+0000 7fdbb178b640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb940a3bb0 0x7fdb940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.765+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdba00cfc40 con 0x7fdbac101160 2024-04-23T18:18:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.765+0000 7fdbb178b640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb940a3bb0 0x7fdb940a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c031b50 tx=0x7fdb9c005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.770+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba009cac0 con 0x7fdbac101160 2024-04-23T18:18:16.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:16 smithi172 ceph-mon[32456]: Deploying daemon mds.h on smithi172 2024-04-23T18:18:16.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:16 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6836/3035435067,v1:172.21.15.173:6837/3035435067] up:boot 2024-04-23T18:18:16.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:16 smithi172 ceph-mon[32456]: fsmap 7 up:standby 2024-04-23T18:18:16.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:16 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:18:16.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:16 smithi173 ceph-mon[32906]: Deploying daemon mds.h on smithi172 2024-04-23T18:18:16.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:16 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6836/3035435067,v1:172.21.15.173:6837/3035435067] up:boot 2024-04-23T18:18:16.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:16 smithi173 ceph-mon[32906]: fsmap 7 up:standby 2024-04-23T18:18:16.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:16 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:18:16.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:16 smithi037 ceph-mon[29404]: Deploying daemon mds.h on smithi172 2024-04-23T18:18:16.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:16 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6836/3035435067,v1:172.21.15.173:6837/3035435067] up:boot 2024-04-23T18:18:16.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:16 smithi037 ceph-mon[29404]: fsmap 7 up:standby 2024-04-23T18:18:16.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:16 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:18:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.997+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbac02d5c0 con 0x7fdbac101160 2024-04-23T18:18:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:16.998+0000 7fdbaa7fc640 1 -- 172.21.15.37:0/1222260665 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7fdba0009de0 con 0x7fdbac101160 2024-04-23T18:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:16.991 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-23T18:18:16.992 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-23T18:18:16.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb940a3bb0 msgr2=0x7fdb940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:16.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb940a3bb0 0x7fdb940a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c031b50 tx=0x7fdb9c005ef0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 msgr2=0x7fdbac19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 0x7fdbac19a970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d950 tx=0x7fdba000de10 comp rx=0 tx=0).stop 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 shutdown_connections 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fdb940a3bb0 0x7fdb940a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbac101b00 0x7fdbac1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbac101160 0x7fdbac19a970 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 --2- 172.21.15.37:0/1222260665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbac100800 0x7fdbac19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 >> 172.21.15.37:0/1222260665 conn(0x7fdbac0fc400 msgr2=0x7fdbac0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 shutdown_connections 2024-04-23T18:18:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:17.000+0000 7fdbb39ef640 1 -- 172.21.15.37:0/1222260665 wait complete. 2024-04-23T18:18:17.317 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 systemd[1]: Starting Ceph mds.h for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:17.577 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 podman[61875]: 2024-04-23 18:18:17.32382902 +0000 UTC m=+0.114679246 container create 96c8aaefce8d05593e2c75f22c76a2ff5ed506417f1ac4ece76e3ccc2713e801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T18:18:17.578 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 podman[61875]: 2024-04-23 18:18:17.227409552 +0000 UTC m=+0.018259741 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:17.578 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 podman[61875]: 2024-04-23 18:18:17.518180321 +0000 UTC m=+0.309030504 container init 96c8aaefce8d05593e2c75f22c76a2ff5ed506417f1ac4ece76e3ccc2713e801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:18:17.578 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 podman[61875]: 2024-04-23 18:18:17.520938046 +0000 UTC m=+0.311788237 container start 96c8aaefce8d05593e2c75f22c76a2ff5ed506417f1ac4ece76e3ccc2713e801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T18:18:17.578 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 ceph-mds[61894]: -- 172.21.15.172:0/117709671 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55582d6a9a40 con 0x55582d67cc00 2024-04-23T18:18:17.578 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h[61890]: starting mds.h at 2024-04-23T18:18:17.578 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 bash[61875]: 96c8aaefce8d05593e2c75f22c76a2ff5ed506417f1ac4ece76e3ccc2713e801 2024-04-23T18:18:17.837 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:18:17 smithi172 systemd[1]: Started Ceph mds.h for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:17.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:17 smithi172 ceph-mon[32456]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:17.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:17 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1222260665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:17.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:17 smithi173 ceph-mon[32906]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:17.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:17 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1222260665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:17.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:17 smithi037 ceph-mon[29404]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:17.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:17 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1222260665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:18.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] up:boot 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: fsmap 8 up:standby 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:18.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:18 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] up:boot 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: fsmap 8 up:standby 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:18.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:18 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:19.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:19.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] up:boot 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: fsmap 8 up:standby 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:19.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:18 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 -- 172.21.15.37:0/137819999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 msgr2=0x7fcc3c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 --2- 172.21.15.37:0/137819999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c109320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009ca0 tx=0x7fcc3402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 -- 172.21.15.37:0/137819999 shutdown_connections 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 --2- 172.21.15.37:0/137819999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c109320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 --2- 172.21.15.37:0/137819999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 0x7fcc3c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 --2- 172.21.15.37:0/137819999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3c100e60 0x7fcc3c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 -- 172.21.15.37:0/137819999 >> 172.21.15.37:0/137819999 conn(0x7fcc3c0fc440 msgr2=0x7fcc3c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 -- 172.21.15.37:0/137819999 shutdown_connections 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.126+0000 7fcc42119640 1 -- 172.21.15.37:0/137819999 wait complete. 2024-04-23T18:18:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 Processor -- start 2024-04-23T18:18:19.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 -- start start 2024-04-23T18:18:19.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3c100e60 0x7fcc3c196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 0x7fcc3c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c10c9f0 con 0x7fcc3c102a00 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c10c870 con 0x7fcc3c100e60 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc42119640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c10cb70 con 0x7fcc3c102040 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c19e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36842/0 (socket says 172.21.15.37:36842) 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc3bfff640 1 -- 172.21.15.37:0/4281457132 learned_addr learned my addr 172.21.15.37:0/4281457132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc3b7fe640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3c100e60 0x7fcc3c196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.127+0000 7fcc3affd640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 0x7fcc3c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3bfff640 1 -- 172.21.15.37:0/4281457132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 msgr2=0x7fcc3c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3bfff640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 0x7fcc3c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3bfff640 1 -- 172.21.15.37:0/4281457132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3c100e60 msgr2=0x7fcc3c196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3bfff640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3c100e60 0x7fcc3c196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3bfff640 1 -- 172.21.15.37:0/4281457132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c19ebc0 con 0x7fcc3c102a00 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3affd640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 0x7fcc3c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc3bfff640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c19e550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcc34031a20 tx=0x7fcc34003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34004240 con 0x7fcc3c102a00 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc34003e10 con 0x7fcc3c102a00 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c19ee50 con 0x7fcc3c102a00 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340305c0 con 0x7fcc3c102a00 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.128+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c19f310 con 0x7fcc3c102a00 2024-04-23T18:18:19.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.129+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c108130 con 0x7fcc3c102a00 2024-04-23T18:18:19.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.130+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc3402f660 con 0x7fcc3c102a00 2024-04-23T18:18:19.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.131+0000 7fcc38ff9640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcc240a3bb0 0x7fcc240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:19.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.131+0000 7fcc3b7fe640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcc240a3bb0 0x7fcc240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:19.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.131+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fcc340f1820 con 0x7fcc3c102a00 2024-04-23T18:18:19.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.132+0000 7fcc3b7fe640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcc240a3bb0 0x7fcc240a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c101bb0 tx=0x7fcc2c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:19.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.136+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc340be6a0 con 0x7fcc3c102a00 2024-04-23T18:18:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.364+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc3c062b50 con 0x7fcc3c102a00 2024-04-23T18:18:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.364+0000 7fcc38ff9640 1 -- 172.21.15.37:0/4281457132 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fcc340bdfa0 con 0x7fcc3c102a00 2024-04-23T18:18:19.358 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:19.358 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:18:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-23T18:18:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.366+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcc240a3bb0 msgr2=0x7fcc240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.366+0000 7fcc42119640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcc240a3bb0 0x7fcc240a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c101bb0 tx=0x7fcc2c00a2d0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.366+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 msgr2=0x7fcc3c19e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.366+0000 7fcc42119640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c19e550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcc34031a20 tx=0x7fcc34003c20 comp rx=0 tx=0).stop 2024-04-23T18:18:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.366+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 shutdown_connections 2024-04-23T18:18:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcc240a3bb0 0x7fcc240a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc3c102a00 0x7fcc3c19e550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc3c102040 0x7fcc3c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 --2- 172.21.15.37:0/4281457132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc3c100e60 0x7fcc3c196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 >> 172.21.15.37:0/4281457132 conn(0x7fcc3c0fc440 msgr2=0x7fcc3c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 shutdown_connections 2024-04-23T18:18:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:19.367+0000 7fcc42119640 1 -- 172.21.15.37:0/4281457132 wait complete. 2024-04-23T18:18:19.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:19 smithi037 ceph-mon[29404]: Deploying daemon mds.i on smithi173 2024-04-23T18:18:19.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:19 smithi037 ceph-mon[29404]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:19.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:19 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4281457132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:19.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:19 smithi173 ceph-mon[32906]: Deploying daemon mds.i on smithi173 2024-04-23T18:18:19.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:19 smithi173 ceph-mon[32906]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:19.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:19 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4281457132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:19.856 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:19 smithi173 systemd[1]: Starting Ceph mds.i for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:20.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:19 smithi172 ceph-mon[32456]: Deploying daemon mds.i on smithi173 2024-04-23T18:18:20.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:19 smithi172 ceph-mon[32456]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:20.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:19 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4281457132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:20.212 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:19 smithi173 podman[63494]: 2024-04-23 18:18:19.869350252 +0000 UTC m=+0.018624336 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:20.212 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:19 smithi173 podman[63494]: 2024-04-23 18:18:19.975274189 +0000 UTC m=+0.124548269 container create f395a5ca45a63f6ed36ad858f2d62841158dc30b179a9ab8b1765ceb3ba755ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T18:18:20.212 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:20 smithi173 podman[63494]: 2024-04-23 18:18:20.21980106 +0000 UTC m=+0.369075140 container init f395a5ca45a63f6ed36ad858f2d62841158dc30b179a9ab8b1765ceb3ba755ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:18:20.605 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:20 smithi173 podman[63494]: 2024-04-23 18:18:20.222532634 +0000 UTC m=+0.371806723 container start f395a5ca45a63f6ed36ad858f2d62841158dc30b179a9ab8b1765ceb3ba755ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-23T18:18:20.605 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:20 smithi173 ceph-mds[63513]: -- 172.21.15.173:0/3103410243 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4e68c9a40 con 0x55c4e689d000 2024-04-23T18:18:20.605 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i[63509]: starting mds.i at 2024-04-23T18:18:20.605 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:20 smithi173 bash[63494]: f395a5ca45a63f6ed36ad858f2d62841158dc30b179a9ab8b1765ceb3ba755ba 2024-04-23T18:18:20.605 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:18:20 smithi173 systemd[1]: Started Ceph mds.i for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:21.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 -- 172.21.15.37:0/328828411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 msgr2=0x7fd67c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 --2- 172.21.15.37:0/328828411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c1036c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009930 tx=0x7fd66c02a420 comp rx=0 tx=0).stop 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 -- 172.21.15.37:0/328828411 shutdown_connections 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 --2- 172.21.15.37:0/328828411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c103c00 0x7fd67c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 --2- 172.21.15.37:0/328828411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c1036c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 --2- 172.21.15.37:0/328828411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67c100270 0x7fd67c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 -- 172.21.15.37:0/328828411 >> 172.21.15.37:0/328828411 conn(0x7fd67c0fbbc0 msgr2=0x7fd67c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 -- 172.21.15.37:0/328828411 shutdown_connections 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.485+0000 7fd683a9a640 1 -- 172.21.15.37:0/328828411 wait complete. 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 Processor -- start 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 -- start start 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c100270 0x7fd67c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67c103c00 0x7fd67c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c10db50 con 0x7fd67c103240 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c10d9d0 con 0x7fd67c100270 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd683a9a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c10dcd0 con 0x7fd67c103c00 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36870/0 (socket says 172.21.15.37:36870) 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 -- 172.21.15.37:0/1400936804 learned_addr learned my addr 172.21.15.37:0/1400936804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:21.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd682037640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67c103c00 0x7fd67c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681836640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c100270 0x7fd67c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 -- 172.21.15.37:0/1400936804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67c103c00 msgr2=0x7fd67c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67c103c00 0x7fd67c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 -- 172.21.15.37:0/1400936804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c100270 msgr2=0x7fd67c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c100270 0x7fd67c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681035640 1 -- 172.21.15.37:0/1400936804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c19fe70 con 0x7fd67c103240 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.486+0000 7fd681836640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c100270 0x7fd67c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.487+0000 7fd681035640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c198790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009a50 tx=0x7fd66c032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.487+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c03f030 con 0x7fd67c103240 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.487+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd66c0044b0 con 0x7fd67c103240 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.487+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c1a0100 con 0x7fd67c103240 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.487+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c035070 con 0x7fd67c103240 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.487+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1a0640 con 0x7fd67c103240 2024-04-23T18:18:21.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.488+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c109410 con 0x7fd67c103240 2024-04-23T18:18:21.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.489+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd66c042030 con 0x7fd67c103240 2024-04-23T18:18:21.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.489+0000 7fd672ffd640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6680a3c30 0x7fd6680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:21.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.490+0000 7fd681836640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6680a3c30 0x7fd6680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:21.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.490+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fd66c0f06a0 con 0x7fd67c103240 2024-04-23T18:18:21.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.490+0000 7fd681836640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6680a3c30 0x7fd6680a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd678005d00 tx=0x7fd678005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:21.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.494+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd66c0bd480 con 0x7fd67c103240 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: Deploying daemon mds.g on smithi037 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] up:boot 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: fsmap 9 up:standby 2024-04-23T18:18:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:21 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:21.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:21.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: Deploying daemon mds.g on smithi037 2024-04-23T18:18:21.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] up:boot 2024-04-23T18:18:21.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: fsmap 9 up:standby 2024-04-23T18:18:21.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:21 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:18:21.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:21.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: Deploying daemon mds.g on smithi037 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] up:boot 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: fsmap 9 up:standby 2024-04-23T18:18:21.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:21 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:18:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.721+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd67c062b50 con 0x7fd67c103240 2024-04-23T18:18:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.722+0000 7fd672ffd640 1 -- 172.21.15.37:0/1400936804 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fd66c0bcd80 con 0x7fd67c103240 2024-04-23T18:18:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:21.715 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:18:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-04-23T18:18:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6680a3c30 msgr2=0x7fd6680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6680a3c30 0x7fd6680a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd678005d00 tx=0x7fd678005ef0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 msgr2=0x7fd67c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c198790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009a50 tx=0x7fd66c032f90 comp rx=0 tx=0).stop 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 shutdown_connections 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6680a3c30 0x7fd6680a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67c103c00 0x7fd67c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd67c103240 0x7fd67c198790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 --2- 172.21.15.37:0/1400936804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd67c100270 0x7fd67c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 >> 172.21.15.37:0/1400936804 conn(0x7fd67c0fbbc0 msgr2=0x7fd67c1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 shutdown_connections 2024-04-23T18:18:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:21.724+0000 7fd683a9a640 1 -- 172.21.15.37:0/1400936804 wait complete. 2024-04-23T18:18:22.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:22 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1400936804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:22.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:22 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1400936804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:22.622 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 systemd[1]: Starting Ceph mds.g for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:22.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:22 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1400936804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:23.044 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 podman[68236]: 2024-04-23 18:18:22.642689666 +0000 UTC m=+0.018889271 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:23.045 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 podman[68236]: 2024-04-23 18:18:22.749105384 +0000 UTC m=+0.125304986 container create a289d6c64445d5520c8938fe544cfa3a5fa2e563f3245fbdb88470919d9b75b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:18:23.045 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 podman[68236]: 2024-04-23 18:18:22.960036786 +0000 UTC m=+0.336236391 container init a289d6c64445d5520c8938fe544cfa3a5fa2e563f3245fbdb88470919d9b75b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T18:18:23.045 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 podman[68236]: 2024-04-23 18:18:22.962824033 +0000 UTC m=+0.339023638 container start a289d6c64445d5520c8938fe544cfa3a5fa2e563f3245fbdb88470919d9b75b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T18:18:23.045 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 ceph-mds[68256]: -- 172.21.15.37:0/688721439 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a29007ba40 con 0x55a29004ec00 2024-04-23T18:18:23.045 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:22 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g[68252]: starting mds.g at 2024-04-23T18:18:23.045 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:23 smithi037 bash[68236]: a289d6c64445d5520c8938fe544cfa3a5fa2e563f3245fbdb88470919d9b75b9 2024-04-23T18:18:23.350 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:18:23 smithi037 systemd[1]: Started Ceph mds.g for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:23.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:23.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:23.606 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:23 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:23.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:23.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:23.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:23.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:23 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:23.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:23.739 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:23 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 -- 172.21.15.37:0/4068821433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33501053a0 msgr2=0x7f3350107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 --2- 172.21.15.37:0/4068821433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33501053a0 0x7f3350107790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f333c009900 tx=0x7f333c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 -- 172.21.15.37:0/4068821433 shutdown_connections 2024-04-23T18:18:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 --2- 172.21.15.37:0/4068821433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3350107cd0 0x7f335010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 --2- 172.21.15.37:0/4068821433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33501053a0 0x7f3350107790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 --2- 172.21.15.37:0/4068821433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3350102a70 0x7f3350104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 -- 172.21.15.37:0/4068821433 >> 172.21.15.37:0/4068821433 conn(0x7f33500fc400 msgr2=0x7f33500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 -- 172.21.15.37:0/4068821433 shutdown_connections 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.842+0000 7f3356ff0640 1 -- 172.21.15.37:0/4068821433 wait complete. 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 Processor -- start 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 -- start start 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3350102a70 0x7f3350198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33501053a0 0x7f3350198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 0x7f335019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f335010dac0 con 0x7f3350107cd0 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f335010d940 con 0x7f3350102a70 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3356ff0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f335010dc40 con 0x7f33501053a0 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 0x7f335019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 0x7f335019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36902/0 (socket says 172.21.15.37:36902) 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 -- 172.21.15.37:0/1126061525 learned_addr learned my addr 172.21.15.37:0/1126061525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f334ffff640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33501053a0 0x7f3350198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f3354d8c640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3350102a70 0x7f3350198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 -- 172.21.15.37:0/1126061525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33501053a0 msgr2=0x7f3350198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33501053a0 0x7f3350198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 -- 172.21.15.37:0/1126061525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3350102a70 msgr2=0x7f3350198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3350102a70 0x7f3350198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 -- 172.21.15.37:0/1126061525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335019fe00 con 0x7f3350107cd0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f334ffff640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33501053a0 0x7f3350198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.843+0000 7f335558d640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 0x7f335019f790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f334800cce0 tx=0x7f334800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.844+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334800f450 con 0x7f3350107cd0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.844+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f334800faa0 con 0x7f3350107cd0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.844+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33501a00f0 con 0x7f3350107cd0 2024-04-23T18:18:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.844+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348018b70 con 0x7f3350107cd0 2024-04-23T18:18:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.844+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33501a05c0 con 0x7f3350107cd0 2024-04-23T18:18:23.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.845+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33500fdef0 con 0x7f3350107cd0 2024-04-23T18:18:23.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.846+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3348018d10 con 0x7f3350107cd0 2024-04-23T18:18:23.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.846+0000 7f334dffb640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f33400a3bb0 0x7f33400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:23.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.846+0000 7f3354d8c640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f33400a3bb0 0x7f33400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:23.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.846+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f33480d0450 con 0x7f3350107cd0 2024-04-23T18:18:23.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.847+0000 7f3354d8c640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f33400a3bb0 0x7f33400a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3344005fd0 tx=0x7f3344005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:23.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:23.851+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3348014030 con 0x7f3350107cd0 2024-04-23T18:18:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.090+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3350062b50 con 0x7f3350107cd0 2024-04-23T18:18:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.091+0000 7f334dffb640 1 -- 172.21.15.37:0/1126061525 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f334809d2d0 con 0x7f3350107cd0 2024-04-23T18:18:24.084 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:24.084 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:18:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 11 2024-04-23T18:18:24.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f33400a3bb0 msgr2=0x7f33400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:24.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f33400a3bb0 0x7f33400a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3344005fd0 tx=0x7f3344005e20 comp rx=0 tx=0).stop 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 msgr2=0x7f335019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 0x7f335019f790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f334800cce0 tx=0x7f334800a620 comp rx=0 tx=0).stop 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 shutdown_connections 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f33400a3bb0 0x7f33400a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3350107cd0 0x7f335019f790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33501053a0 0x7f3350198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 --2- 172.21.15.37:0/1126061525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3350102a70 0x7f3350198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 >> 172.21.15.37:0/1126061525 conn(0x7f33500fc400 msgr2=0x7f33501034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 shutdown_connections 2024-04-23T18:18:24.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:24.093+0000 7f3356ff0640 1 -- 172.21.15.37:0/1126061525 wait complete. 2024-04-23T18:18:24.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:24 smithi037 ceph-mon[29404]: Deploying daemon mds.a on smithi037 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:24 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.37:6838/514853653,v1:172.21.15.37:6839/514853653] up:boot 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:24 smithi037 ceph-mon[29404]: fsmap 10 up:standby 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:24 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:24 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1126061525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:24.754 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:24 smithi172 ceph-mon[32456]: Deploying daemon mds.a on smithi037 2024-04-23T18:18:24.755 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:24 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.37:6838/514853653,v1:172.21.15.37:6839/514853653] up:boot 2024-04-23T18:18:24.755 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:24 smithi172 ceph-mon[32456]: fsmap 10 up:standby 2024-04-23T18:18:24.755 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:24 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:18:24.755 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:24 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1126061525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:24.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:24 smithi173 ceph-mon[32906]: Deploying daemon mds.a on smithi037 2024-04-23T18:18:24.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:24 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.37:6838/514853653,v1:172.21.15.37:6839/514853653] up:boot 2024-04-23T18:18:24.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:24 smithi173 ceph-mon[32906]: fsmap 10 up:standby 2024-04-23T18:18:24.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:24 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:18:24.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:24 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1126061525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:25.618 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:25 smithi037 ceph-mon[29404]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:25.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:25 smithi172 ceph-mon[32456]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:25.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:25 smithi173 ceph-mon[32906]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:26.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:26.120 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:18:25 smithi037 systemd[1]: Started Ceph mds.a for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 -- 172.21.15.37:0/1010634185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610101fc0 msgr2=0x7f76101023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 --2- 172.21.15.37:0/1010634185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610101fc0 0x7f76101023e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f760c009900 tx=0x7f760c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 -- 172.21.15.37:0/1010634185 shutdown_connections 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 --2- 172.21.15.37:0/1010634185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7610103b70 0x7f761010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 --2- 172.21.15.37:0/1010634185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 0x7f7610103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 --2- 172.21.15.37:0/1010634185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610101fc0 0x7f76101023e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 -- 172.21.15.37:0/1010634185 >> 172.21.15.37:0/1010634185 conn(0x7f76100fd550 msgr2=0x7f76100ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 -- 172.21.15.37:0/1010634185 shutdown_connections 2024-04-23T18:18:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.211+0000 7f76174e1640 1 -- 172.21.15.37:0/1010634185 wait complete. 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 Processor -- start 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 -- start start 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7610101fc0 0x7f7610075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 0x7f7610074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 0x7f7610074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f761010db00 con 0x7f7610103b70 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f761010d980 con 0x7f7610101fc0 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f76174e1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f761010dc80 con 0x7f76101031b0 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f7615a7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 0x7f7610074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f7615a7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 0x7f7610074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60656/0 (socket says 172.21.15.37:60656) 2024-04-23T18:18:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f7615a7e640 1 -- 172.21.15.37:0/2227483130 learned_addr learned my addr 172.21.15.37:0/2227483130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.212+0000 7f761527d640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7610101fc0 0x7f7610075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7614a7c640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 0x7f7610074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7615a7e640 1 -- 172.21.15.37:0/2227483130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 msgr2=0x7f7610074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7615a7e640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 0x7f7610074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7615a7e640 1 -- 172.21.15.37:0/2227483130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7610101fc0 msgr2=0x7f7610075cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7615a7e640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7610101fc0 0x7f7610075cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7615a7e640 1 -- 172.21.15.37:0/2227483130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7610075140 con 0x7f7610103b70 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7614a7c640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 0x7f7610074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f7615a7e640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 0x7f7610074880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f760800eaa0 tx=0x7f760800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760800cdb0 con 0x7f7610103b70 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608005020 con 0x7f7610103b70 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76101a23a0 con 0x7f7610103b70 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76080116d0 con 0x7f7610103b70 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.213+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76101a28c0 con 0x7f7610103b70 2024-04-23T18:18:26.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.214+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7610109450 con 0x7f7610103b70 2024-04-23T18:18:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.215+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76080051d0 con 0x7f7610103b70 2024-04-23T18:18:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.216+0000 7f76067fc640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f75fc0a3c30 0x7f75fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.216+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f76080d0460 con 0x7f7610103b70 2024-04-23T18:18:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.216+0000 7f761527d640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f75fc0a3c30 0x7f75fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.216+0000 7f761527d640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f75fc0a3c30 0x7f75fc0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f760c009230 tx=0x7f760c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:26.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.220+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7608016030 con 0x7f7610103b70 2024-04-23T18:18:26.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.446+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f761002d5c0 con 0x7f7610103b70 2024-04-23T18:18:26.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.446+0000 7f76067fc640 1 -- 172.21.15.37:0/2227483130 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f760809d360 con 0x7f7610103b70 2024-04-23T18:18:26.440 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:26.440 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:18:26.440 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.448+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f75fc0a3c30 msgr2=0x7f75fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.448+0000 7f76174e1640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f75fc0a3c30 0x7f75fc0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f760c009230 tx=0x7f760c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.448+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 msgr2=0x7f7610074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.448+0000 7f76174e1640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 0x7f7610074880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f760800eaa0 tx=0x7f760800ef60 comp rx=0 tx=0).stop 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 shutdown_connections 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f75fc0a3c30 0x7f75fc0a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610103b70 0x7f7610074880 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76101031b0 0x7f7610074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 --2- 172.21.15.37:0/2227483130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7610101fc0 0x7f7610075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 >> 172.21.15.37:0/2227483130 conn(0x7f76100fd550 msgr2=0x7f7610106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:26.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 shutdown_connections 2024-04-23T18:18:26.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:26.449+0000 7f76174e1640 1 -- 172.21.15.37:0/2227483130 wait complete. 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: Deploying daemon mds.b on smithi172 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.37:6840/2155547418,v1:172.21.15.37:6841/2155547418] up:boot 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: fsmap 11 up:standby 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:18:27.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:26 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2227483130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:27.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:27.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: Deploying daemon mds.b on smithi172 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.37:6840/2155547418,v1:172.21.15.37:6841/2155547418] up:boot 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: fsmap 11 up:standby 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:18:27.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:26 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2227483130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: Deploying daemon mds.b on smithi172 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.37:6840/2155547418,v1:172.21.15.37:6841/2155547418] up:boot 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: fsmap 11 up:standby 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:18:27.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:26 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2227483130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:28.303 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 systemd[1]: Starting Ceph mds.b for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:18:28.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 -- 172.21.15.37:0/2842577279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c107d10 msgr2=0x7ff08c10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 --2- 172.21.15.37:0/2842577279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c107d10 0x7ff08c10a100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009900 tx=0x7ff07c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 -- 172.21.15.37:0/2842577279 shutdown_connections 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 --2- 172.21.15.37:0/2842577279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c107d10 0x7ff08c10a100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 --2- 172.21.15.37:0/2842577279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff08c1053e0 0x7ff08c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 --2- 172.21.15.37:0/2842577279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102ab0 0x7ff08c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 -- 172.21.15.37:0/2842577279 >> 172.21.15.37:0/2842577279 conn(0x7ff08c0fc400 msgr2=0x7ff08c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 -- 172.21.15.37:0/2842577279 shutdown_connections 2024-04-23T18:18:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.576+0000 7ff090c43640 1 -- 172.21.15.37:0/2842577279 wait complete. 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 Processor -- start 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 -- start start 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff08c102ab0 0x7ff08c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 0x7ff08c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c107d10 0x7ff08c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c10daf0 con 0x7ff08c1053e0 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c10d970 con 0x7ff08c102ab0 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff090c43640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c10dc70 con 0x7ff08c107d10 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff089d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 0x7ff08c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff089d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 0x7ff08c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60676/0 (socket says 172.21.15.37:60676) 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff089d9b640 1 -- 172.21.15.37:0/540876443 learned_addr learned my addr 172.21.15.37:0/540876443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff08ad9d640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c107d10 0x7ff08c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.577+0000 7ff08a59c640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff08c102ab0 0x7ff08c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff089d9b640 1 -- 172.21.15.37:0/540876443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c107d10 msgr2=0x7ff08c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff089d9b640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c107d10 0x7ff08c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff089d9b640 1 -- 172.21.15.37:0/540876443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff08c102ab0 msgr2=0x7ff08c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff089d9b640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff08c102ab0 0x7ff08c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff089d9b640 1 -- 172.21.15.37:0/540876443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c19fdc0 con 0x7ff08c1053e0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff08ad9d640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c107d10 0x7ff08c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff089d9b640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 0x7ff08c1986e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e9a0 tx=0x7ff07400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07400cd80 con 0x7ff08c1053e0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c1a00b0 con 0x7ff08c1053e0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0740035f0 con 0x7ff08c1053e0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074010690 con 0x7ff08c1053e0 2024-04-23T18:18:28.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.578+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a0600 con 0x7ff08c1053e0 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 podman[62195]: 2024-04-23 18:18:28.310385373 +0000 UTC m=+0.111355210 container create 7aabb44825f8180487b0b211fb2ebb38af44d90ae6e5d33c8020bb5278cc14c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 podman[62195]: 2024-04-23 18:18:28.217301035 +0000 UTC m=+0.018270839 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 podman[62195]: 2024-04-23 18:18:28.504618554 +0000 UTC m=+0.305588349 container init 7aabb44825f8180487b0b211fb2ebb38af44d90ae6e5d33c8020bb5278cc14c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 podman[62195]: 2024-04-23 18:18:28.507293617 +0000 UTC m=+0.308263413 container start 7aabb44825f8180487b0b211fb2ebb38af44d90ae6e5d33c8020bb5278cc14c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 ceph-mds[62214]: -- 172.21.15.172:0/209936406 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d9ab2ba40 con 0x561d9aafec00 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: starting mds.b at 2024-04-23T18:18:28.575 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 bash[62195]: 7aabb44825f8180487b0b211fb2ebb38af44d90ae6e5d33c8020bb5278cc14c4 2024-04-23T18:18:28.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.579+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c0fdef0 con 0x7ff08c1053e0 2024-04-23T18:18:28.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.580+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff074003fd0 con 0x7ff08c1053e0 2024-04-23T18:18:28.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.581+0000 7ff0837fe640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff0780a3bb0 0x7ff0780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:28.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.581+0000 7ff08a59c640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff0780a3bb0 0x7ff0780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:28.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.581+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff0740d0310 con 0x7ff08c1053e0 2024-04-23T18:18:28.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.581+0000 7ff08a59c640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff0780a3bb0 0x7ff0780a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff084005fd0 tx=0x7ff084009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:28.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.585+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff07409d190 con 0x7ff08c1053e0 2024-04-23T18:18:28.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.816+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08c02d5c0 con 0x7ff08c1053e0 2024-04-23T18:18:28.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.817+0000 7ff0837fe640 1 -- 172.21.15.37:0/540876443 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7ff07409ca90 con 0x7ff08c1053e0 2024-04-23T18:18:28.811 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:28.811 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T18:18:28.811 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-04-23T18:18:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff0780a3bb0 msgr2=0x7ff0780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff0780a3bb0 0x7ff0780a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff084005fd0 tx=0x7ff084009500 comp rx=0 tx=0).stop 2024-04-23T18:18:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 msgr2=0x7ff08c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 0x7ff08c1986e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e9a0 tx=0x7ff07400ee60 comp rx=0 tx=0).stop 2024-04-23T18:18:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 shutdown_connections 2024-04-23T18:18:28.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff0780a3bb0 0x7ff0780a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c107d10 0x7ff08c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff08c1053e0 0x7ff08c1986e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 --2- 172.21.15.37:0/540876443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff08c102ab0 0x7ff08c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 >> 172.21.15.37:0/540876443 conn(0x7ff08c0fc400 msgr2=0x7ff08c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 shutdown_connections 2024-04-23T18:18:28.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:28.820+0000 7ff090c43640 1 -- 172.21.15.37:0/540876443 wait complete. 2024-04-23T18:18:28.837 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:28 smithi172 systemd[1]: Started Ceph mds.b for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/540876443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] up:boot 2024-04-23T18:18:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: fsmap 12 up:standby 2024-04-23T18:18:29.839 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:29 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:18:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/540876443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] up:boot 2024-04-23T18:18:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: fsmap 12 up:standby 2024-04-23T18:18:29.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:29 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/540876443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] up:boot 2024-04-23T18:18:29.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: fsmap 12 up:standby 2024-04-23T18:18:29.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:29 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:18:30.825 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- 172.21.15.37:0/289024053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90101fe0 msgr2=0x7fce90102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- 172.21.15.37:0/289024053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90101fe0 0x7fce90102460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fce78009900 tx=0x7fce7802a410 comp rx=0 tx=0).stop 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- 172.21.15.37:0/289024053 shutdown_connections 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- 172.21.15.37:0/289024053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce901029a0 0x7fce901092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- 172.21.15.37:0/289024053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90101fe0 0x7fce90102460 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- 172.21.15.37:0/289024053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90100e90 0x7fce901012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- 172.21.15.37:0/289024053 >> 172.21.15.37:0/289024053 conn(0x7fce900fc400 msgr2=0x7fce900fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- 172.21.15.37:0/289024053 shutdown_connections 2024-04-23T18:18:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- 172.21.15.37:0/289024053 wait complete. 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 Processor -- start 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- start start 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 0x7fce901981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90101fe0 0x7fce90198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce901029a0 0x7fce9019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010db30 con 0x7fce90100e90 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010d9b0 con 0x7fce90101fe0 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.938+0000 7fce9679b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce9010dcb0 con 0x7fce901029a0 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 0x7fce901981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 0x7fce901981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60692/0 (socket says 172.21.15.37:60692) 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 -- 172.21.15.37:0/2007480620 learned_addr learned my addr 172.21.15.37:0/2007480620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce87fff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90101fe0 0x7fce90198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce94d38640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce901029a0 0x7fce9019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:30.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 -- 172.21.15.37:0/2007480620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce901029a0 msgr2=0x7fce9019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce901029a0 0x7fce9019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 -- 172.21.15.37:0/2007480620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90101fe0 msgr2=0x7fce90198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90101fe0 0x7fce90198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 -- 172.21.15.37:0/2007480620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9019fdf0 con 0x7fce90100e90 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce87fff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90101fe0 0x7fce90198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8ffff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 0x7fce901981d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce8000e9f0 tx=0x7fce8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8000cd80 con 0x7fce90100e90 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce800108e0 con 0x7fce90100e90 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a00e0 con 0x7fce90100e90 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80018980 con 0x7fce90100e90 2024-04-23T18:18:30.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.939+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a0650 con 0x7fce90100e90 2024-04-23T18:18:30.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.940+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce901081c0 con 0x7fce90100e90 2024-04-23T18:18:30.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.941+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fce80010430 con 0x7fce90100e90 2024-04-23T18:18:30.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.942+0000 7fce8dffb640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fce7c0a3c30 0x7fce7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:30.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.942+0000 7fce87fff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fce7c0a3c30 0x7fce7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:30.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.942+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fce800d0220 con 0x7fce90100e90 2024-04-23T18:18:30.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.942+0000 7fce87fff640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fce7c0a3c30 0x7fce7c0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce78009230 tx=0x7fce780314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:30.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:30.947+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce8009ca50 con 0x7fce90100e90 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:30 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.194+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce9002d5c0 con 0x7fce90100e90 2024-04-23T18:18:31.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.195+0000 7fce8dffb640 1 -- 172.21.15.37:0/2007480620 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fce800a0d50 con 0x7fce90100e90 2024-04-23T18:18:31.188 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:31.188 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-23T18:18:31.189 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-04-23T18:18:31.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fce7c0a3c30 msgr2=0x7fce7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fce7c0a3c30 0x7fce7c0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce78009230 tx=0x7fce780314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 msgr2=0x7fce901981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 0x7fce901981d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce8000e9f0 tx=0x7fce8000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 shutdown_connections 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fce7c0a3c30 0x7fce7c0a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce901029a0 0x7fce9019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce90101fe0 0x7fce90198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 --2- 172.21.15.37:0/2007480620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce90100e90 0x7fce901981d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 >> 172.21.15.37:0/2007480620 conn(0x7fce900fc400 msgr2=0x7fce90105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 shutdown_connections 2024-04-23T18:18:31.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.197+0000 7fce9679b640 1 -- 172.21.15.37:0/2007480620 wait complete. 2024-04-23T18:18:31.203 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:18:31.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 -- 172.21.15.37:0/2966386569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48101fe0 msgr2=0x7f9e48102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/2966386569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48101fe0 0x7f9e48102460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009900 tx=0x7f9e3402a410 comp rx=0 tx=0).stop 2024-04-23T18:18:31.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 -- 172.21.15.37:0/2966386569 shutdown_connections 2024-04-23T18:18:31.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/2966386569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e481029a0 0x7f9e481092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/2966386569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48101fe0 0x7f9e48102460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/2966386569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e48100e90 0x7f9e481012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 -- 172.21.15.37:0/2966386569 >> 172.21.15.37:0/2966386569 conn(0x7f9e480fc400 msgr2=0x7f9e480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 -- 172.21.15.37:0/2966386569 shutdown_connections 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 -- 172.21.15.37:0/2966386569 wait complete. 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 Processor -- start 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.314+0000 7f9e4ec05640 1 -- start start 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 0x7f9e48069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e48101fe0 0x7f9e4819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e481029a0 0x7f9e4819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4810c9d0 con 0x7f9e48100e90 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4810c850 con 0x7f9e481029a0 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4810cb50 con 0x7f9e48101fe0 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 0x7f9e48069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 0x7f9e48069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60708/0 (socket says 172.21.15.37:60708) 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 -- 172.21.15.37:0/4107289295 learned_addr learned my addr 172.21.15.37:0/4107289295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4d1a2640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e481029a0 0x7f9e4819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e47fff640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e48101fe0 0x7f9e4819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 -- 172.21.15.37:0/4107289295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e48101fe0 msgr2=0x7f9e4819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e48101fe0 0x7f9e4819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 -- 172.21.15.37:0/4107289295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e481029a0 msgr2=0x7f9e4819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e481029a0 0x7f9e4819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 -- 172.21.15.37:0/4107289295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4819fe80 con 0x7f9e48100e90 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4c9a1640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 0x7f9e48069c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00d950 tx=0x7f9e3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c017070 con 0x7f9e48100e90 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.315+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e481a0110 con 0x7f9e48100e90 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.316+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3c009c30 con 0x7f9e48100e90 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.316+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00f620 con 0x7f9e48100e90 2024-04-23T18:18:31.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.316+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e481a0680 con 0x7f9e48100e90 2024-04-23T18:18:31.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.317+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e48108210 con 0x7f9e48100e90 2024-04-23T18:18:31.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.318+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e3c010460 con 0x7f9e48100e90 2024-04-23T18:18:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.318+0000 7f9e45ffb640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9e380a3bb0 0x7f9e380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.319+0000 7f9e47fff640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9e380a3bb0 0x7f9e380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.319+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f9e3c0cfae0 con 0x7f9e48100e90 2024-04-23T18:18:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.319+0000 7f9e47fff640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9e380a3bb0 0x7f9e380a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009230 tx=0x7f9e340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:31.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.323+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e3c009de0 con 0x7f9e48100e90 2024-04-23T18:18:31.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:30 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:30 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.511+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e4802d800 con 0x7f9e48100e90 2024-04-23T18:18:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.513+0000 7f9e45ffb640 1 -- 172.21.15.37:0/4107289295 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18480 (secure 0 0 0) 0x7f9e3c09c310 con 0x7f9e48100e90 2024-04-23T18:18:31.506 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:17:48.804072+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:18:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9e380a3bb0 msgr2=0x7f9e380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9e380a3bb0 0x7f9e380a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009230 tx=0x7f9e340314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 msgr2=0x7f9e48069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 0x7f9e48069c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00d950 tx=0x7f9e3c00de10 comp rx=0 tx=0).stop 2024-04-23T18:18:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 shutdown_connections 2024-04-23T18:18:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9e380a3bb0 0x7f9e380a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e481029a0 0x7f9e4819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e48101fe0 0x7f9e4819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 --2- 172.21.15.37:0/4107289295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e48100e90 0x7f9e48069c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 >> 172.21.15.37:0/4107289295 conn(0x7f9e480fc400 msgr2=0x7f9e481053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 shutdown_connections 2024-04-23T18:18:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.515+0000 7f9e4ec05640 1 -- 172.21.15.37:0/4107289295 wait complete. 2024-04-23T18:18:31.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:18:31.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:18:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 -- 172.21.15.37:0/3481302086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 msgr2=0x7f1110101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 --2- 172.21.15.37:0/3481302086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f1110101280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1100009900 tx=0x7f110002a410 comp rx=0 tx=0).stop 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 -- 172.21.15.37:0/3481302086 shutdown_connections 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 --2- 172.21.15.37:0/3481302086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f1110109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 --2- 172.21.15.37:0/3481302086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 0x7f11101024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 --2- 172.21.15.37:0/3481302086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f1110101280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 -- 172.21.15.37:0/3481302086 >> 172.21.15.37:0/3481302086 conn(0x7f11100fc440 msgr2=0x7f11100fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 -- 172.21.15.37:0/3481302086 shutdown_connections 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 -- 172.21.15.37:0/3481302086 wait complete. 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 Processor -- start 2024-04-23T18:18:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.635+0000 7f1116631640 1 -- start start 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1116631640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f111019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1116631640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 0x7f111019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1116631640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f11101a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1116631640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f111010db30 con 0x7f1110100e60 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1116631640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f111010d9b0 con 0x7f1110102040 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1116631640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f111010dcb0 con 0x7f1110102a00 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f11101a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f110ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f111019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f110f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 0x7f111019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f11101a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:36336/0 (socket says 172.21.15.37:36336) 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f110f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 0x7f111019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:42880/0 (socket says 172.21.15.37:42880) 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 -- 172.21.15.37:0/2326702559 learned_addr learned my addr 172.21.15.37:0/2326702559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 -- 172.21.15.37:0/2326702559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 msgr2=0x7f111019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 0x7f111019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 -- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 msgr2=0x7f111019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f111019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f1114bce640 1 -- 172.21.15.37:0/2326702559 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11101a2070 con 0x7f1110102a00 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.636+0000 7f110ffff640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f111019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.637+0000 7f1114bce640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f11101a1a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f110800eb10 tx=0x7f110800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.637+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108004b20 con 0x7f1110102a00 2024-04-23T18:18:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.637+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11101a2360 con 0x7f1110102a00 2024-04-23T18:18:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.637+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1108004cc0 con 0x7f1110102a00 2024-04-23T18:18:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.637+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108010690 con 0x7f1110102a00 2024-04-23T18:18:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.637+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11101a2900 con 0x7f1110102a00 2024-04-23T18:18:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.638+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11101a2c20 con 0x7f1110102a00 2024-04-23T18:18:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.640+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1108005020 con 0x7f1110102a00 2024-04-23T18:18:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.640+0000 7f110d7fa640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f10f80a3bb0 0x7f10f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.641+0000 7f110ffff640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f10f80a3bb0 0x7f10f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.641+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f11080d0500 con 0x7f1110102a00 2024-04-23T18:18:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.641+0000 7f110ffff640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f10f80a3bb0 0x7f10f80a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1100009230 tx=0x7f11000314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.644+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f110809d2e0 con 0x7f1110102a00 2024-04-23T18:18:31.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.853+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f11101a2df0 con 0x7f1110102a00 2024-04-23T18:18:31.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.854+0000 7f110d7fa640 1 -- 172.21.15.37:0/2326702559 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f110809cbe0 con 0x7f1110102a00 2024-04-23T18:18:31.847 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:18:31.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f10f80a3bb0 msgr2=0x7f10f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f10f80a3bb0 0x7f10f80a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1100009230 tx=0x7f11000314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 msgr2=0x7f11101a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f11101a1a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f110800eb10 tx=0x7f110800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 shutdown_connections 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f10f80a3bb0 0x7f10f80a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1110102a00 0x7f11101a1a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1110102040 0x7f111019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 --2- 172.21.15.37:0/2326702559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1110100e60 0x7f111019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 >> 172.21.15.37:0/2326702559 conn(0x7f11100fc440 msgr2=0x7f1110107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 shutdown_connections 2024-04-23T18:18:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.856+0000 7f1116631640 1 -- 172.21.15.37:0/2326702559 wait complete. 2024-04-23T18:18:31.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 -- 172.21.15.37:0/3686364044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 msgr2=0x7efc441092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 --2- 172.21.15.37:0/3686364044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc441092c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009ca0 tx=0x7efc3c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 -- 172.21.15.37:0/3686364044 shutdown_connections 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 --2- 172.21.15.37:0/3686364044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc441092c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 --2- 172.21.15.37:0/3686364044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc44101fe0 0x7efc44102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 --2- 172.21.15.37:0/3686364044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc44100e90 0x7efc441012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 -- 172.21.15.37:0/3686364044 >> 172.21.15.37:0/3686364044 conn(0x7efc440fc440 msgr2=0x7efc440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 -- 172.21.15.37:0/3686364044 shutdown_connections 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.972+0000 7efc496e1640 1 -- 172.21.15.37:0/3686364044 wait complete. 2024-04-23T18:18:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 Processor -- start 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 -- start start 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc44100e90 0x7efc4410b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc44101fe0 0x7efc44109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc4410a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410db30 con 0x7efc441029a0 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410d9b0 con 0x7efc44101fe0 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc496e1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410dcb0 con 0x7efc44100e90 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc4410a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc4410a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60732/0 (socket says 172.21.15.37:60732) 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc437fe640 1 -- 172.21.15.37:0/2469526883 learned_addr learned my addr 172.21.15.37:0/2469526883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc427fc640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc44101fe0 0x7efc44109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc42ffd640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc44100e90 0x7efc4410b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.973+0000 7efc437fe640 1 -- 172.21.15.37:0/2469526883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc44100e90 msgr2=0x7efc4410b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc437fe640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc44100e90 0x7efc4410b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc437fe640 1 -- 172.21.15.37:0/2469526883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc44101fe0 msgr2=0x7efc44109e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc437fe640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc44101fe0 0x7efc44109e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc437fe640 1 -- 172.21.15.37:0/2469526883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4410ac40 con 0x7efc441029a0 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc437fe640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc4410a3b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097b0 tx=0x7efc3c031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003b70 con 0x7efc441029a0 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3c003d10 con 0x7efc441029a0 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc441a2280 con 0x7efc441029a0 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c02fc10 con 0x7efc441029a0 2024-04-23T18:18:31.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.974+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc441a27d0 con 0x7efc441029a0 2024-04-23T18:18:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.975+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc441081a0 con 0x7efc441029a0 2024-04-23T18:18:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.976+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc3c004040 con 0x7efc441029a0 2024-04-23T18:18:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.977+0000 7efc2bfff640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.977+0000 7efc42ffd640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.977+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7efc3c0f14e0 con 0x7efc441029a0 2024-04-23T18:18:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.977+0000 7efc42ffd640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efc34009bd0 tx=0x7efc34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:31.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:31.981+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc3c0be2c0 con 0x7efc441029a0 2024-04-23T18:18:32.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2007480620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4107289295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:31 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2326702559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2007480620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4107289295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.133 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:31 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2326702559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.178+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efc44062b50 con 0x7efc441029a0 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.178+0000 7efc2bfff640 1 -- 172.21.15.37:0/2469526883 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7efc3c0bdbc0 con 0x7efc441029a0 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:18:32.172 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7efc2c0a3bb0 msgr2=0x7efc2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efc34009bd0 tx=0x7efc34005f90 comp rx=0 tx=0).stop 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 msgr2=0x7efc4410a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc4410a3b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097b0 tx=0x7efc3c031770 comp rx=0 tx=0).stop 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 shutdown_connections 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7efc2c0a3bb0 0x7efc2c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441029a0 0x7efc4410a3b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc44101fe0 0x7efc44109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 --2- 172.21.15.37:0/2469526883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc44100e90 0x7efc4410b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 >> 172.21.15.37:0/2469526883 conn(0x7efc440fc440 msgr2=0x7efc44105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 shutdown_connections 2024-04-23T18:18:32.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.180+0000 7efc496e1640 1 -- 172.21.15.37:0/2469526883 wait complete. 2024-04-23T18:18:32.184 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T18:18:32.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 -- 172.21.15.37:0/3752400293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 msgr2=0x7f507c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 --2- 172.21.15.37:0/3752400293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c102460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f506c009900 tx=0x7f506c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 -- 172.21.15.37:0/3752400293 shutdown_connections 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 --2- 172.21.15.37:0/3752400293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 0x7f507c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 --2- 172.21.15.37:0/3752400293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c102460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 --2- 172.21.15.37:0/3752400293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f507c100e90 0x7f507c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 -- 172.21.15.37:0/3752400293 >> 172.21.15.37:0/3752400293 conn(0x7f507c0fc400 msgr2=0x7f507c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:32.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 -- 172.21.15.37:0/3752400293 shutdown_connections 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.300+0000 7f5084322640 1 -- 172.21.15.37:0/3752400293 wait complete. 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 Processor -- start 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 -- start start 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f507c100e90 0x7f507c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 0x7f507c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f507c10c9d0 con 0x7f507c101fe0 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f507c10c850 con 0x7f507c100e90 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f5084322640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f507c10cb50 con 0x7f507c1029a0 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:32.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60754/0 (socket says 172.21.15.37:60754) 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 -- 172.21.15.37:0/231423219 learned_addr learned my addr 172.21.15.37:0/231423219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50820be640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f507c100e90 0x7f507c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50828bf640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 0x7f507c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 -- 172.21.15.37:0/231423219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 msgr2=0x7f507c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 0x7f507c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 -- 172.21.15.37:0/231423219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f507c100e90 msgr2=0x7f507c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f507c100e90 0x7f507c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 -- 172.21.15.37:0/231423219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c19fe40 con 0x7f507c101fe0 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50828bf640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 0x7f507c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.301+0000 7f50818bd640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c198760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f506c006010 tx=0x7f506c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:32.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.302+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c03f030 con 0x7f507c101fe0 2024-04-23T18:18:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.302+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f506c003cb0 con 0x7f507c101fe0 2024-04-23T18:18:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.302+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c1a00d0 con 0x7f507c101fe0 2024-04-23T18:18:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.302+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c035070 con 0x7f507c101fe0 2024-04-23T18:18:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.302+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c1a05c0 con 0x7f507c101fe0 2024-04-23T18:18:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.303+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507c1a0820 con 0x7f507c101fe0 2024-04-23T18:18:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.304+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f506c042030 con 0x7f507c101fe0 2024-04-23T18:18:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.304+0000 7f50737fe640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f50680a3bb0 0x7f50680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.305+0000 7f50820be640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f50680a3bb0 0x7f50680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.305+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f506c0f0a40 con 0x7f507c101fe0 2024-04-23T18:18:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.305+0000 7f50820be640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f50680a3bb0 0x7f50680a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5078005d00 tx=0x7f507800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:32.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.308+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f506c0bd820 con 0x7f507c101fe0 2024-04-23T18:18:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:32.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:32.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2007480620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4107289295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:32.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:31 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2326702559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:18:32.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:32.511+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f507c1a0c10 con 0x7f507c101fe0 2024-04-23T18:18:33.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.024+0000 7f50737fe640 1 -- 172.21.15.37:0/231423219 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f506c0bd120 con 0x7f507c101fe0 2024-04-23T18:18:33.018 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.026+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f50680a3bb0 msgr2=0x7f50680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.026+0000 7f5084322640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f50680a3bb0 0x7f50680a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5078005d00 tx=0x7f507800b410 comp rx=0 tx=0).stop 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.026+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 msgr2=0x7f507c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.026+0000 7f5084322640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c198760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f506c006010 tx=0x7f506c0042f0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 shutdown_connections 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f50680a3bb0 0x7f50680a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f507c1029a0 0x7f507c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f507c101fe0 0x7f507c198760 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 --2- 172.21.15.37:0/231423219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f507c100e90 0x7f507c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 >> 172.21.15.37:0/231423219 conn(0x7f507c0fc400 msgr2=0x7f507c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 shutdown_connections 2024-04-23T18:18:33.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.027+0000 7f5084322640 1 -- 172.21.15.37:0/231423219 wait complete. 2024-04-23T18:18:33.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T18:18:33.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.149+0000 7fc36d115640 1 -- 172.21.15.37:0/3087350552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102040 msgr2=0x7fc3681024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:33.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.149+0000 7fc36d115640 1 --2- 172.21.15.37:0/3087350552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102040 0x7fc3681024c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009900 tx=0x7fc35c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:33.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 -- 172.21.15.37:0/3087350552 shutdown_connections 2024-04-23T18:18:33.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 --2- 172.21.15.37:0/3087350552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc368102a00 0x7fc368109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 --2- 172.21.15.37:0/3087350552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102040 0x7fc3681024c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 --2- 172.21.15.37:0/3087350552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc368100e60 0x7fc368101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 -- 172.21.15.37:0/3087350552 >> 172.21.15.37:0/3087350552 conn(0x7fc3680fc440 msgr2=0x7fc3680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 -- 172.21.15.37:0/3087350552 shutdown_connections 2024-04-23T18:18:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 -- 172.21.15.37:0/3087350552 wait complete. 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.150+0000 7fc36d115640 1 Processor -- start 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 -- start start 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc368100e60 0x7fc368198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc368102040 0x7fc368198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 0x7fc36819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc36810ca00 con 0x7fc368102a00 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc36810c880 con 0x7fc368100e60 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36d115640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc36810cb80 con 0x7fc368102040 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 0x7fc36819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc366d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc368100e60 0x7fc368198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 0x7fc36819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60764/0 (socket says 172.21.15.37:60764) 2024-04-23T18:18:33.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 -- 172.21.15.37:0/2756803002 learned_addr learned my addr 172.21.15.37:0/2756803002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36659c640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc368102040 0x7fc368198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 -- 172.21.15.37:0/2756803002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc368102040 msgr2=0x7fc368198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc368102040 0x7fc368198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 -- 172.21.15.37:0/2756803002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc368100e60 msgr2=0x7fc368198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc368100e60 0x7fc368198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 -- 172.21.15.37:0/2756803002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36819fe20 con 0x7fc368102a00 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc36759e640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 0x7fc36819f7b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc36000eaa0 tx=0x7fc36000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.151+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36000cdb0 con 0x7fc368102a00 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.152+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3681a0110 con 0x7fc368102a00 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.152+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc360005020 con 0x7fc368102a00 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.152+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3600116d0 con 0x7fc368102a00 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.152+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3681a0680 con 0x7fc368102a00 2024-04-23T18:18:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.153+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3681a0940 con 0x7fc368102a00 2024-04-23T18:18:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.154+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3600054a0 con 0x7fc368102a00 2024-04-23T18:18:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.155+0000 7fc34ffff640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc3500a3bb0 0x7fc3500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.155+0000 7fc366d9d640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc3500a3bb0 0x7fc3500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.155+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7fc3600d0570 con 0x7fc368102a00 2024-04-23T18:18:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.155+0000 7fc366d9d640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc3500a3bb0 0x7fc3500a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc368101bb0 tx=0x7fc35800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.159+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc360016030 con 0x7fc368102a00 2024-04-23T18:18:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:33.344+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc3681a0b10 con 0x7fc368102a00 2024-04-23T18:18:33.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2469526883' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/231423219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:33.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:33.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2469526883' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:18:33.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/231423219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:18:33.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:33.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2469526883' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/231423219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:18:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.030+0000 7fc34ffff640 1 -- 172.21.15.37:0/2756803002 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7fc36009cc30 con 0x7fc368102a00 2024-04-23T18:18:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-23T18:18:34.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc3500a3bb0 msgr2=0x7fc3500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:34.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc3500a3bb0 0x7fc3500a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc368101bb0 tx=0x7fc35800b390 comp rx=0 tx=0).stop 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 msgr2=0x7fc36819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 0x7fc36819f7b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc36000eaa0 tx=0x7fc36000ef60 comp rx=0 tx=0).stop 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 shutdown_connections 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc3500a3bb0 0x7fc3500a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc368102a00 0x7fc36819f7b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc368102040 0x7fc368198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 --2- 172.21.15.37:0/2756803002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc368100e60 0x7fc368198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 >> 172.21.15.37:0/2756803002 conn(0x7fc3680fc440 msgr2=0x7fc3680fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 shutdown_connections 2024-04-23T18:18:34.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.034+0000 7fc36d115640 1 -- 172.21.15.37:0/2756803002 wait complete. 2024-04-23T18:18:34.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T18:18:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 -- 172.21.15.37:0/2926045742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c103190 msgr2=0x7fd76c105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 --2- 172.21.15.37:0/2926045742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c103190 0x7fd76c105600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009900 tx=0x7fd75c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 -- 172.21.15.37:0/2926045742 shutdown_connections 2024-04-23T18:18:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 --2- 172.21.15.37:0/2926045742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd76c105b40 0x7fd76c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 --2- 172.21.15.37:0/2926045742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c103190 0x7fd76c105600 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 --2- 172.21.15.37:0/2926045742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c100820 0x7fd76c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 -- 172.21.15.37:0/2926045742 >> 172.21.15.37:0/2926045742 conn(0x7fd76c0fc440 msgr2=0x7fd76c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 -- 172.21.15.37:0/2926045742 shutdown_connections 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 -- 172.21.15.37:0/2926045742 wait complete. 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.170+0000 7fd771137640 1 Processor -- start 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 -- start start 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 0x7fd76c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c103190 0x7fd76c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd76c105b40 0x7fd76c1a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c1140e0 con 0x7fd76c100820 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c113f60 con 0x7fd76c105b40 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd771137640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c114260 con 0x7fd76c103190 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 0x7fd76c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:34.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 0x7fd76c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60782/0 (socket says 172.21.15.37:60782) 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 -- 172.21.15.37:0/2690264535 learned_addr learned my addr 172.21.15.37:0/2690264535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76b59e640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd76c105b40 0x7fd76c1a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76a59c640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c103190 0x7fd76c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 -- 172.21.15.37:0/2690264535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c103190 msgr2=0x7fd76c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c103190 0x7fd76c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 -- 172.21.15.37:0/2690264535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd76c105b40 msgr2=0x7fd76c1a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd76c105b40 0x7fd76c1a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 -- 172.21.15.37:0/2690264535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c1a63d0 con 0x7fd76c100820 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76a59c640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c103190 0x7fd76c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd76ad9d640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 0x7fd76c19e7b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c970 tx=0x7fd76000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.171+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760007bb0 con 0x7fd76c100820 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.172+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c1a66c0 con 0x7fd76c100820 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.172+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd760007d50 con 0x7fd76c100820 2024-04-23T18:18:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.172+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76000f450 con 0x7fd76c100820 2024-04-23T18:18:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.172+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a6bc0 con 0x7fd76c100820 2024-04-23T18:18:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.173+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c0fdf30 con 0x7fd76c100820 2024-04-23T18:18:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.174+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd760004f20 con 0x7fd76c100820 2024-04-23T18:18:34.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.174+0000 7fd753fff640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd7540a3bb0 0x7fd7540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:34.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.175+0000 7fd76a59c640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd7540a3bb0 0x7fd7540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:34.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.175+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7fd7600d0060 con 0x7fd76c100820 2024-04-23T18:18:34.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.175+0000 7fd76a59c640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd7540a3bb0 0x7fd7540a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd76c19fcc0 tx=0x7fd75c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:34.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.179+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd760014030 con 0x7fd76c100820 2024-04-23T18:18:34.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:34 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/231423219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:18:34.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:34 smithi172 ceph-mon[32456]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:18:34.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:34 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2756803002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:18:34.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:34 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/231423219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:18:34.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:34 smithi173 ceph-mon[32906]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:18:34.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:34 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2756803002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:18:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/231423219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:18:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:34 smithi037 ceph-mon[29404]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:18:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2756803002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:18:34.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:34.407+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd76c02d5c0 con 0x7fd76c100820 2024-04-23T18:18:35.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.034+0000 7fd753fff640 1 -- 172.21.15.37:0/2690264535 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd76009cb50 con 0x7fd76c100820 2024-04-23T18:18:35.028 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T18:18:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.037+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd7540a3bb0 msgr2=0x7fd7540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.037+0000 7fd771137640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd7540a3bb0 0x7fd7540a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd76c19fcc0 tx=0x7fd75c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.037+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 msgr2=0x7fd76c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.037+0000 7fd771137640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 0x7fd76c19e7b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c970 tx=0x7fd76000ce30 comp rx=0 tx=0).stop 2024-04-23T18:18:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.038+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 shutdown_connections 2024-04-23T18:18:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.038+0000 7fd771137640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd7540a3bb0 0x7fd7540a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.038+0000 7fd771137640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd76c105b40 0x7fd76c1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.038+0000 7fd771137640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd76c103190 0x7fd76c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.038+0000 7fd771137640 1 --2- 172.21.15.37:0/2690264535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd76c100820 0x7fd76c19e7b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.038+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 >> 172.21.15.37:0/2690264535 conn(0x7fd76c0fc440 msgr2=0x7fd76c101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:35.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.039+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 shutdown_connections 2024-04-23T18:18:35.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.039+0000 7fd771137640 1 -- 172.21.15.37:0/2690264535 wait complete. 2024-04-23T18:18:35.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:18:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 -- 172.21.15.37:0/392107081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 msgr2=0x7f2a0c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 --2- 172.21.15.37:0/392107081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c1092d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009ca0 tx=0x7f2a0402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 -- 172.21.15.37:0/392107081 shutdown_connections 2024-04-23T18:18:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 --2- 172.21.15.37:0/392107081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c1092d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 --2- 172.21.15.37:0/392107081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a0c101ff0 0x7f2a0c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 --2- 172.21.15.37:0/392107081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c100ea0 0x7f2a0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 -- 172.21.15.37:0/392107081 >> 172.21.15.37:0/392107081 conn(0x7f2a0c0fc430 msgr2=0x7f2a0c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 -- 172.21.15.37:0/392107081 shutdown_connections 2024-04-23T18:18:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 -- 172.21.15.37:0/392107081 wait complete. 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.175+0000 7f2a10b68640 1 Processor -- start 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 -- start start 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a0c100ea0 0x7f2a0c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c101ff0 0x7f2a0c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10c9e0 con 0x7f2a0c1029b0 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10c860 con 0x7f2a0c100ea0 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a10b68640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10cb60 con 0x7f2a0c101ff0 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42962/0 (socket says 172.21.15.37:42962) 2024-04-23T18:18:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 -- 172.21.15.37:0/1016857337 learned_addr learned my addr 172.21.15.37:0/1016857337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0a59c640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a0c100ea0 0x7f2a0c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a09d9b640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c101ff0 0x7f2a0c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 -- 172.21.15.37:0/1016857337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c101ff0 msgr2=0x7f2a0c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c101ff0 0x7f2a0c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 -- 172.21.15.37:0/1016857337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a0c100ea0 msgr2=0x7f2a0c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a0c100ea0 0x7f2a0c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 -- 172.21.15.37:0/1016857337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19fd00 con 0x7f2a0c1029b0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a09d9b640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c101ff0 0x7f2a0c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.176+0000 7f2a0ad9d640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c19f690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a04031a20 tx=0x7f2a04003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.177+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04004240 con 0x7f2a0c1029b0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.177+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c19ff90 con 0x7f2a0c1029b0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.177+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a04003e10 con 0x7f2a0c1029b0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.177+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a040305c0 con 0x7f2a0c1029b0 2024-04-23T18:18:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.177+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a04a0 con 0x7f2a0c1029b0 2024-04-23T18:18:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.178+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c1a0700 con 0x7f2a0c1029b0 2024-04-23T18:18:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.179+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a0402f660 con 0x7f2a0c1029b0 2024-04-23T18:18:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.179+0000 7f2a037fe640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f29f80a3bb0 0x7f29f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.180+0000 7f2a0a59c640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f29f80a3bb0 0x7f29f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.180+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f2a040f1c90 con 0x7f2a0c1029b0 2024-04-23T18:18:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.180+0000 7f2a0a59c640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f29f80a3bb0 0x7f29f80a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0058c0 tx=0x7f29fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.183+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a040be7c0 con 0x7f2a0c1029b0 2024-04-23T18:18:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:35 smithi172 ceph-mon[32456]: pgmap v282: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:35 smithi172 ceph-mon[32456]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:18:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2756803002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:18:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:35 smithi172 ceph-mon[32456]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:18:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2690264535' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:18:35.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:35 smithi172 ceph-mon[32456]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:18:35.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:35 smithi173 ceph-mon[32906]: pgmap v282: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:35.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:35 smithi173 ceph-mon[32906]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:18:35.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2756803002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:18:35.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:35 smithi173 ceph-mon[32906]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:18:35.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2690264535' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:18:35.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:35 smithi173 ceph-mon[32906]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:18:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.369+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a0c1a09f0 con 0x7f2a0c1029b0 2024-04-23T18:18:35.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.370+0000 7f2a037fe640 1 -- 172.21.15.37:0/1016857337 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22256 (secure 0 0 0) 0x7f2a040be0c0 con 0x7f2a0c1029b0 2024-04-23T18:18:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:35.364 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:18:34.408153+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:18:35.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f29f80a3bb0 msgr2=0x7f29f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f29f80a3bb0 0x7f29f80a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0058c0 tx=0x7f29fc00a2d0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 msgr2=0x7f2a0c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c19f690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a04031a20 tx=0x7f2a04003c20 comp rx=0 tx=0).stop 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 shutdown_connections 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f29f80a3bb0 0x7f29f80a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a0c1029b0 0x7f2a0c19f690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a0c101ff0 0x7f2a0c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 --2- 172.21.15.37:0/1016857337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a0c100ea0 0x7f2a0c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.372+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 >> 172.21.15.37:0/1016857337 conn(0x7f2a0c0fc430 msgr2=0x7f2a0c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.373+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 shutdown_connections 2024-04-23T18:18:35.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.373+0000 7f2a10b68640 1 -- 172.21.15.37:0/1016857337 wait complete. 2024-04-23T18:18:35.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:35 smithi037 ceph-mon[29404]: pgmap v282: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:35 smithi037 ceph-mon[29404]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2756803002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:35 smithi037 ceph-mon[29404]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2690264535' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:35 smithi037 ceph-mon[29404]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:18:35.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 -- 172.21.15.37:0/2316480938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc102a30 msgr2=0x7efffc104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 --2- 172.21.15.37:0/2316480938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc102a30 0x7efffc104e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7effe4009900 tx=0x7effe402a410 comp rx=0 tx=0).stop 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 -- 172.21.15.37:0/2316480938 shutdown_connections 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 --2- 172.21.15.37:0/2316480938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efffc107c90 0x7efffc10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 --2- 172.21.15.37:0/2316480938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc105360 0x7efffc107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 --2- 172.21.15.37:0/2316480938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc102a30 0x7efffc104e20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 -- 172.21.15.37:0/2316480938 >> 172.21.15.37:0/2316480938 conn(0x7efffc0fc440 msgr2=0x7efffc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 -- 172.21.15.37:0/2316480938 shutdown_connections 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.486+0000 7f0002720640 1 -- 172.21.15.37:0/2316480938 wait complete. 2024-04-23T18:18:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 Processor -- start 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 -- start start 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc102a30 0x7efffc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 0x7efffc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efffc107c90 0x7efffc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efffc10db40 con 0x7efffc105360 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efffc10d9c0 con 0x7efffc102a30 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7f0002720640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efffc10dcc0 con 0x7efffc107c90 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.487+0000 7efffb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 0x7efffc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 0x7efffc1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42978/0 (socket says 172.21.15.37:42978) 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 -- 172.21.15.37:0/81085133 learned_addr learned my addr 172.21.15.37:0/81085133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffbfff640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc102a30 0x7efffc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7f0000cbd640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efffc107c90 0x7efffc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 -- 172.21.15.37:0/81085133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efffc107c90 msgr2=0x7efffc19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efffc107c90 0x7efffc19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 -- 172.21.15.37:0/81085133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc102a30 msgr2=0x7efffc1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc102a30 0x7efffc1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 -- 172.21.15.37:0/81085133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc19fdd0 con 0x7efffc105360 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffbfff640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc102a30 0x7efffc1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efffb7fe640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 0x7efffc1986f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7effec00c970 tx=0x7effec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec014070 con 0x7efffc105360 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec004440 con 0x7efffc105360 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc1a00c0 con 0x7efffc105360 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec003e50 con 0x7efffc105360 2024-04-23T18:18:35.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.488+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a0610 con 0x7efffc105360 2024-04-23T18:18:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.489+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc0fdf30 con 0x7efffc105360 2024-04-23T18:18:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.490+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7effec017070 con 0x7efffc105360 2024-04-23T18:18:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.491+0000 7efff97fa640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7effe80a3d50 0x7effe80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.491+0000 7efffbfff640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7effe80a3d50 0x7effe80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.491+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7effec0cff80 con 0x7efffc105360 2024-04-23T18:18:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.492+0000 7efffbfff640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7effe80a3d50 0x7effe80a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7effe4000c00 tx=0x7effe40314c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.496+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effec01e030 con 0x7efffc105360 2024-04-23T18:18:35.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.681+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efffc062b50 con 0x7efffc105360 2024-04-23T18:18:35.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.682+0000 7efff97fa640 1 -- 172.21.15.37:0/81085133 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22256 (secure 0 0 0) 0x7effec00de20 con 0x7efffc105360 2024-04-23T18:18:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:18:34.408153+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7effe80a3d50 msgr2=0x7effe80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7effe80a3d50 0x7effe80a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7effe4000c00 tx=0x7effe40314c0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 msgr2=0x7efffc1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 0x7efffc1986f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7effec00c970 tx=0x7effec00ce30 comp rx=0 tx=0).stop 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 shutdown_connections 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7effe80a3d50 0x7effe80a6210 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efffc107c90 0x7efffc19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efffc105360 0x7efffc1986f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 --2- 172.21.15.37:0/81085133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efffc102a30 0x7efffc1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 >> 172.21.15.37:0/81085133 conn(0x7efffc0fc440 msgr2=0x7efffc103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.684+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 shutdown_connections 2024-04-23T18:18:35.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.685+0000 7f0002720640 1 -- 172.21.15.37:0/81085133 wait complete. 2024-04-23T18:18:35.689 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.800+0000 7f6e6ac64640 1 -- 172.21.15.37:0/3006900580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 msgr2=0x7f6e64102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.800+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/3006900580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64102460 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009900 tx=0x7f6e4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- 172.21.15.37:0/3006900580 shutdown_connections 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/3006900580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e641029a0 0x7f6e641092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/3006900580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64102460 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/3006900580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e64100e90 0x7f6e641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- 172.21.15.37:0/3006900580 >> 172.21.15.37:0/3006900580 conn(0x7f6e640fc400 msgr2=0x7f6e640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- 172.21.15.37:0/3006900580 shutdown_connections 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- 172.21.15.37:0/3006900580 wait complete. 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 Processor -- start 2024-04-23T18:18:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- start start 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e64100e90 0x7f6e64198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641029a0 0x7f6e6419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6410db30 con 0x7f6e64101fe0 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6410d9b0 con 0x7f6e641029a0 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e6ac64640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6410dcb0 con 0x7f6e64100e90 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42984/0 (socket says 172.21.15.37:42984) 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.801+0000 7f6e63fff640 1 -- 172.21.15.37:0/2727508320 learned_addr learned my addr 172.21.15.37:0/2727508320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e69201640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641029a0 0x7f6e6419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e64100e90 0x7f6e64198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e63fff640 1 -- 172.21.15.37:0/2727508320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e64100e90 msgr2=0x7f6e64198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e63fff640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e64100e90 0x7f6e64198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e63fff640 1 -- 172.21.15.37:0/2727508320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641029a0 msgr2=0x7f6e6419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e63fff640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641029a0 0x7f6e6419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e63fff640 1 -- 172.21.15.37:0/2727508320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6419fe40 con 0x7f6e64101fe0 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e64100e90 0x7f6e64198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e63fff640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64198760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006010 tx=0x7f6e4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c03f030 con 0x7f6e64101fe0 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e4c003cb0 con 0x7f6e64101fe0 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e641a00d0 con 0x7f6e64101fe0 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c035070 con 0x7f6e64101fe0 2024-04-23T18:18:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.802+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e641a05c0 con 0x7f6e64101fe0 2024-04-23T18:18:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.803+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e64108270 con 0x7f6e64101fe0 2024-04-23T18:18:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.804+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e4c042030 con 0x7f6e64101fe0 2024-04-23T18:18:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.805+0000 7f6e61ffb640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6e500a3bb0 0x7f6e500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.805+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6e500a3bb0 0x7f6e500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.805+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f6e4c0f0990 con 0x7f6e64101fe0 2024-04-23T18:18:35.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.806+0000 7f6e68a00640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6e500a3bb0 0x7f6e500a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e54005fd0 tx=0x7f6e540093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:35.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:35.810+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e4c0bd420 con 0x7f6e64101fe0 2024-04-23T18:18:36.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.038+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6e64062b50 con 0x7f6e64101fe0 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2690264535' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1016857337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/81085133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] up:active 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:36 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2727508320' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:18:36.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2690264535' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1016857337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/81085133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] up:active 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:36.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:36 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2727508320' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:18:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2690264535' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1016857337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/81085133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] up:active 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:36.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:36 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2727508320' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:18:36.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.847+0000 7f6e61ffb640 1 -- 172.21.15.37:0/2727508320 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f6e4c0bcd20 con 0x7f6e64101fe0 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6e500a3bb0 msgr2=0x7f6e500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6e500a3bb0 0x7f6e500a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e54005fd0 tx=0x7f6e540093e0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 msgr2=0x7f6e64198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64198760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006010 tx=0x7f6e4c0042f0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 shutdown_connections 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6e500a3bb0 0x7f6e500a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641029a0 0x7f6e6419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e64101fe0 0x7f6e64198760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 --2- 172.21.15.37:0/2727508320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e64100e90 0x7f6e64198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.850+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 >> 172.21.15.37:0/2727508320 conn(0x7f6e640fc400 msgr2=0x7f6e64105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.851+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 shutdown_connections 2024-04-23T18:18:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.851+0000 7f6e6ac64640 1 -- 172.21.15.37:0/2727508320 wait complete. 2024-04-23T18:18:36.857 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-23T18:18:36.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/2007106945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8105340 msgr2=0x7fc0f8107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/2007106945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8105340 0x7fc0f8107730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009900 tx=0x7fc0ec02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/2007106945 shutdown_connections 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/2007106945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0f8107c70 0x7fc0f810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/2007106945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8105340 0x7fc0f8107730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/2007106945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8102a10 0x7fc0f8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/2007106945 >> 172.21.15.37:0/2007106945 conn(0x7fc0f80fc400 msgr2=0x7fc0f80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/2007106945 shutdown_connections 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.975+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/2007106945 wait complete. 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 Processor -- start 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 -- start start 2024-04-23T18:18:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 0x7fc0f8102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8105340 0x7fc0f81009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0f8107c70 0x7fc0f8100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f810db10 con 0x7fc0f8102a10 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f810d990 con 0x7fc0f8105340 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0fd4ae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f810dc90 con 0x7fc0f8107c70 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 0x7fc0f8102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 0x7fc0f8102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42998/0 (socket says 172.21.15.37:42998) 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.976+0000 7fc0f6ffd640 1 -- 172.21.15.37:0/3365915576 learned_addr learned my addr 172.21.15.37:0/3365915576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f67fc640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8105340 0x7fc0f81009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f77fe640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0f8107c70 0x7fc0f8100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f6ffd640 1 -- 172.21.15.37:0/3365915576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0f8107c70 msgr2=0x7fc0f8100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f6ffd640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0f8107c70 0x7fc0f8100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f6ffd640 1 -- 172.21.15.37:0/3365915576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8105340 msgr2=0x7fc0f81009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f6ffd640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8105340 0x7fc0f81009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f6ffd640 1 -- 172.21.15.37:0/3365915576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f8101790 con 0x7fc0f8102a10 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f67fc640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8105340 0x7fc0f81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0f6ffd640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 0x7fc0f8102380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c890 tx=0x7fc0e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8004220 con 0x7fc0f8102a10 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0e80043c0 con 0x7fc0f8102a10 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f81a23c0 con 0x7fc0f8102a10 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8004a00 con 0x7fc0f8102a10 2024-04-23T18:18:36.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.977+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f81a2960 con 0x7fc0f8102a10 2024-04-23T18:18:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.978+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f8068d70 con 0x7fc0f8102a10 2024-04-23T18:18:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.979+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc0e80079b0 con 0x7fc0f8102a10 2024-04-23T18:18:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.980+0000 7fc0dffff640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc0e00a3bb0 0x7fc0e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.980+0000 7fc0f67fc640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc0e00a3bb0 0x7fc0e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.980+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc0e80d01a0 con 0x7fc0f8102a10 2024-04-23T18:18:36.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.980+0000 7fc0f67fc640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc0e00a3bb0 0x7fc0e00a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000c00 tx=0x7fc0ec0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:36.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:36.984+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0e8014030 con 0x7fc0f8102a10 2024-04-23T18:18:37.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:37 smithi037 ceph-mon[29404]: pgmap v285: 97 pgs: 27 active+clean, 37 creating+peering, 33 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:37 smithi037 ceph-mon[29404]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:37 smithi037 ceph-mon[29404]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:37 smithi037 ceph-mon[29404]: Cluster is now healthy 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:37 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2727508320' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:37 smithi037 ceph-mon[29404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:37.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:37.213+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc0f802d950 con 0x7fc0f8102a10 2024-04-23T18:18:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:37 smithi172 ceph-mon[32456]: pgmap v285: 97 pgs: 27 active+clean, 37 creating+peering, 33 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-04-23T18:18:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:37 smithi172 ceph-mon[32456]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:18:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:37 smithi172 ceph-mon[32456]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:18:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:37 smithi172 ceph-mon[32456]: Cluster is now healthy 2024-04-23T18:18:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:37 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2727508320' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:18:37.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:37 smithi172 ceph-mon[32456]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:37 smithi173 ceph-mon[32906]: pgmap v285: 97 pgs: 27 active+clean, 37 creating+peering, 33 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-04-23T18:18:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:37 smithi173 ceph-mon[32906]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:18:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:37 smithi173 ceph-mon[32906]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:18:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:37 smithi173 ceph-mon[32906]: Cluster is now healthy 2024-04-23T18:18:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:37 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2727508320' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:18:37.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:37 smithi173 ceph-mon[32906]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.058+0000 7fc0dffff640 1 -- 172.21.15.37:0/3365915576 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fc0e809cc30 con 0x7fc0f8102a10 2024-04-23T18:18:38.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc0e00a3bb0 msgr2=0x7fc0e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc0e00a3bb0 0x7fc0e00a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000c00 tx=0x7fc0ec0314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 msgr2=0x7fc0f8102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 0x7fc0f8102380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c890 tx=0x7fc0e800cd50 comp rx=0 tx=0).stop 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 shutdown_connections 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc0e00a3bb0 0x7fc0e00a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0f8107c70 0x7fc0f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0f8105340 0x7fc0f81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 --2- 172.21.15.37:0/3365915576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8102a10 0x7fc0f8102380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 >> 172.21.15.37:0/3365915576 conn(0x7fc0f80fc400 msgr2=0x7fc0f8103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 shutdown_connections 2024-04-23T18:18:38.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.061+0000 7fc0fd4ae640 1 -- 172.21.15.37:0/3365915576 wait complete. 2024-04-23T18:18:38.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-23T18:18:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 -- 172.21.15.37:0/3172550092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06941031a0 msgr2=0x7f0694103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 --2- 172.21.15.37:0/3172550092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06941031a0 0x7f0694103620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0688009900 tx=0x7f068802a410 comp rx=0 tx=0).stop 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 -- 172.21.15.37:0/3172550092 shutdown_connections 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 --2- 172.21.15.37:0/3172550092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0694103b60 0x7f069410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 --2- 172.21.15.37:0/3172550092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06941031a0 0x7f0694103620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 --2- 172.21.15.37:0/3172550092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 0x7f06941023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 -- 172.21.15.37:0/3172550092 >> 172.21.15.37:0/3172550092 conn(0x7f06940fd520 msgr2=0x7f06940ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 -- 172.21.15.37:0/3172550092 shutdown_connections 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.189+0000 7f0698e35640 1 -- 172.21.15.37:0/3172550092 wait complete. 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 Processor -- start 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 -- start start 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 0x7f06941981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06941031a0 0x7f0694198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 0x7f069419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f069410db40 con 0x7f0694103b60 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f069410d9c0 con 0x7f0694101fb0 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0698e35640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f069410dcc0 con 0x7f06941031a0 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 0x7f069419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 0x7f069419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43018/0 (socket says 172.21.15.37:43018) 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 -- 172.21.15.37:0/1862013919 learned_addr learned my addr 172.21.15.37:0/1862013919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0691d9b640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06941031a0 0x7f0694198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f069259c640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 0x7f06941981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 -- 172.21.15.37:0/1862013919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06941031a0 msgr2=0x7f0694198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06941031a0 0x7f0694198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 -- 172.21.15.37:0/1862013919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 msgr2=0x7f06941981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 0x7f06941981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 -- 172.21.15.37:0/1862013919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069419fe10 con 0x7f0694103b60 2024-04-23T18:18:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f069259c640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 0x7f06941981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.190+0000 7f0692d9d640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 0x7f069419f7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f068c00cad0 tx=0x7f068c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.191+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c00ae20 con 0x7f0694103b60 2024-04-23T18:18:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.191+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06941a0100 con 0x7f0694103b60 2024-04-23T18:18:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.191+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f068c005020 con 0x7f0694103b60 2024-04-23T18:18:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.191+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c00f6b0 con 0x7f0694103b60 2024-04-23T18:18:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.191+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06941a05d0 con 0x7f0694103b60 2024-04-23T18:18:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.192+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06941093c0 con 0x7f0694103b60 2024-04-23T18:18:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.193+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f068c0051d0 con 0x7f0694103b60 2024-04-23T18:18:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.193+0000 7f06837fe640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f067c0a3bb0 0x7f067c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.194+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f068c0d0370 con 0x7f0694103b60 2024-04-23T18:18:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.194+0000 7f069259c640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f067c0a3bb0 0x7f067c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.194+0000 7f069259c640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f067c0a3bb0 0x7f067c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0694103000 tx=0x7f068400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:38.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.198+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f068c014030 con 0x7f0694103b60 2024-04-23T18:18:38.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:38 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3365915576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-23T18:18:38.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3365915576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-23T18:18:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3365915576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-23T18:18:38.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:38.428+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f069402d5c0 con 0x7f0694103b60 2024-04-23T18:18:39.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.065+0000 7f06837fe640 1 -- 172.21.15.37:0/1862013919 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f068c09cea0 con 0x7f0694103b60 2024-04-23T18:18:39.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f067c0a3bb0 msgr2=0x7f067c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:39.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f067c0a3bb0 0x7f067c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0694103000 tx=0x7f068400a560 comp rx=0 tx=0).stop 2024-04-23T18:18:39.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 msgr2=0x7f069419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 0x7f069419f7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f068c00cad0 tx=0x7f068c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 shutdown_connections 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f067c0a3bb0 0x7f067c0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0694103b60 0x7f069419f7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06941031a0 0x7f0694198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 --2- 172.21.15.37:0/1862013919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101fb0 0x7f06941981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 >> 172.21.15.37:0/1862013919 conn(0x7f06940fd520 msgr2=0x7f0694106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 shutdown_connections 2024-04-23T18:18:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.068+0000 7f0698e35640 1 -- 172.21.15.37:0/1862013919 wait complete. 2024-04-23T18:18:39.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 -- 172.21.15.37:0/227665904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40105360 msgr2=0x7f9b40107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/227665904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40105360 0x7f9b40107750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009900 tx=0x7f9b3402a410 comp rx=0 tx=0).stop 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 -- 172.21.15.37:0/227665904 shutdown_connections 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/227665904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 0x7f9b4010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/227665904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40105360 0x7f9b40107750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/227665904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b40102a30 0x7f9b40104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 -- 172.21.15.37:0/227665904 >> 172.21.15.37:0/227665904 conn(0x7f9b400fc440 msgr2=0x7f9b400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 -- 172.21.15.37:0/227665904 shutdown_connections 2024-04-23T18:18:39.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.195+0000 7f9b45ee1640 1 -- 172.21.15.37:0/227665904 wait complete. 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 Processor -- start 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 -- start start 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 0x7f9b40198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b40105360 0x7f9b40198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 0x7f9b4019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4010db40 con 0x7f9b40102a30 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4010d9c0 con 0x7f9b40107c90 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b45ee1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4010dcc0 con 0x7f9b40105360 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 0x7f9b40198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 0x7f9b40198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43028/0 (socket says 172.21.15.37:43028) 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 -- 172.21.15.37:0/4223492374 learned_addr learned my addr 172.21.15.37:0/4223492374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3ffff640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 0x7f9b4019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3effd640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b40105360 0x7f9b40198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 -- 172.21.15.37:0/4223492374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b40105360 msgr2=0x7f9b40198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b40105360 0x7f9b40198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 -- 172.21.15.37:0/4223492374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 msgr2=0x7f9b4019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 0x7f9b4019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.196+0000 7f9b3f7fe640 1 -- 172.21.15.37:0/4223492374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4019fe30 con 0x7f9b40102a30 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b3ffff640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 0x7f9b4019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b3f7fe640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 0x7f9b40198210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000d950 tx=0x7f9b3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30017070 con 0x7f9b40102a30 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b30009c30 con 0x7f9b40102a30 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b401a0120 con 0x7f9b40102a30 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000f620 con 0x7f9b40102a30 2024-04-23T18:18:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.197+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b401a0640 con 0x7f9b40102a30 2024-04-23T18:18:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.198+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b400fdf30 con 0x7f9b40102a30 2024-04-23T18:18:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.199+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b30010460 con 0x7f9b40102a30 2024-04-23T18:18:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.200+0000 7f9b3cff9640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9b280a3bb0 0x7f9b280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.200+0000 7f9b3effd640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9b280a3bb0 0x7f9b280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.200+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9b300cfee0 con 0x7f9b40102a30 2024-04-23T18:18:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.200+0000 7f9b3effd640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9b280a3bb0 0x7f9b280a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006010 tx=0x7f9b340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.205+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b3009ca10 con 0x7f9b40102a30 2024-04-23T18:18:39.337 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:18:38 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: 2024-04-23T18:18:38.893+0000 7f057b92c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: pgmap v287: 97 pgs: 49 active+clean, 37 creating+peering, 11 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3365915576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T18:18:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1862013919' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:18:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: pgmap v287: 97 pgs: 49 active+clean, 37 creating+peering, 11 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-23T18:18:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:18:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3365915576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-23T18:18:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:18:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:39.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T18:18:39.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T18:18:39.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1862013919' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:18:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: pgmap v287: 97 pgs: 49 active+clean, 37 creating+peering, 11 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-23T18:18:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:18:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3365915576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1862013919' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:18:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:39.435+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f9b4002d5c0 con 0x7f9b40102a30 2024-04-23T18:18:40.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.080+0000 7f9b3cff9640 1 -- 172.21.15.37:0/4223492374 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f9b30009de0 con 0x7f9b40102a30 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9b280a3bb0 msgr2=0x7f9b280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9b280a3bb0 0x7f9b280a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006010 tx=0x7f9b340314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 msgr2=0x7f9b40198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 0x7f9b40198210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000d950 tx=0x7f9b3000de10 comp rx=0 tx=0).stop 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 shutdown_connections 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9b280a3bb0 0x7f9b280a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b40107c90 0x7f9b4019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b40105360 0x7f9b40198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 --2- 172.21.15.37:0/4223492374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b40102a30 0x7f9b40198210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.082+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 >> 172.21.15.37:0/4223492374 conn(0x7f9b400fc440 msgr2=0x7f9b40103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:40.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.083+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 shutdown_connections 2024-04-23T18:18:40.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.083+0000 7f9b45ee1640 1 -- 172.21.15.37:0/4223492374 wait complete. 2024-04-23T18:18:40.090 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-23T18:18:40.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-23T18:18:40.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 -- 172.21.15.37:0/3225944727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 msgr2=0x7f93d01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 --2- 172.21.15.37:0/3225944727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01023d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009900 tx=0x7f93cc02a410 comp rx=0 tx=0).stop 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 -- 172.21.15.37:0/3225944727 shutdown_connections 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 --2- 172.21.15.37:0/3225944727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 0x7f93d010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 --2- 172.21.15.37:0/3225944727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93d01031a0 0x7f93d0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 --2- 172.21.15.37:0/3225944727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01023d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 -- 172.21.15.37:0/3225944727 >> 172.21.15.37:0/3225944727 conn(0x7f93d00fd520 msgr2=0x7f93d00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 -- 172.21.15.37:0/3225944727 shutdown_connections 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 -- 172.21.15.37:0/3225944727 wait complete. 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.209+0000 7f93d8205640 1 Processor -- start 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 -- start start 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93d01031a0 0x7f93d0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 0x7f93d019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93d010dab0 con 0x7f93d0101fb0 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93d010d930 con 0x7f93d0103b60 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d8205640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93d010dc30 con 0x7f93d01031a0 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43042/0 (socket says 172.21.15.37:43042) 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 -- 172.21.15.37:0/536288932 learned_addr learned my addr 172.21.15.37:0/536288932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d67a2640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 0x7f93d019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d57a0640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93d01031a0 0x7f93d0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 -- 172.21.15.37:0/536288932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93d01031a0 msgr2=0x7f93d0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93d01031a0 0x7f93d0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 -- 172.21.15.37:0/536288932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 msgr2=0x7f93d019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 0x7f93d019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 -- 172.21.15.37:0/536288932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d019fe00 con 0x7f93d0101fb0 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d67a2640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 0x7f93d019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:40.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93d5fa1640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01981e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93cc006010 tx=0x7f93cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:40.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.210+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03f030 con 0x7f93d0101fb0 2024-04-23T18:18:40.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.211+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc0318f0 con 0x7f93d0101fb0 2024-04-23T18:18:40.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.211+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d01a0090 con 0x7f93d0101fb0 2024-04-23T18:18:40.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.211+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc035070 con 0x7f93d0101fb0 2024-04-23T18:18:40.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.211+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d01a0580 con 0x7f93d0101fb0 2024-04-23T18:18:40.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.212+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d01093a0 con 0x7f93d0101fb0 2024-04-23T18:18:40.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.213+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93cc042030 con 0x7f93d0101fb0 2024-04-23T18:18:40.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.213+0000 7f93c6ffd640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f93bc0a3bb0 0x7f93bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.213+0000 7f93d57a0640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f93bc0a3bb0 0x7f93bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.213+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f93cc0f0810 con 0x7f93d0101fb0 2024-04-23T18:18:40.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.214+0000 7f93d57a0640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f93bc0a3bb0 0x7f93bc0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93d01996f0 tx=0x7f93c000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:40.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.218+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93cc0bd340 con 0x7f93d0101fb0 2024-04-23T18:18:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1862013919' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] up:active 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: Cluster is now healthy 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-23T18:18:40.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:40 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4223492374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:18:40.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1862013919' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:18:40.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] up:active 2024-04-23T18:18:40.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:18:40.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:18:40.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: Cluster is now healthy 2024-04-23T18:18:40.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T18:18:40.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-23T18:18:40.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-23T18:18:40.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:40 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4223492374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:18:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1862013919' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:18:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] up:active 2024-04-23T18:18:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:18:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:18:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: Cluster is now healthy 2024-04-23T18:18:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T18:18:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-23T18:18:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-23T18:18:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4223492374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:18:40.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.431+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f93d0062390 con 0x7f93bc0a3bb0 2024-04-23T18:18:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.457+0000 7f93c6ffd640 1 -- 172.21.15.37:0/536288932 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f93d0062390 con 0x7f93bc0a3bb0 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f93bc0a3bb0 msgr2=0x7f93bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f93bc0a3bb0 0x7f93bc0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93d01996f0 tx=0x7f93c000a4c0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 msgr2=0x7f93d01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01981e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93cc006010 tx=0x7f93cc0042f0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 shutdown_connections 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f93bc0a3bb0 0x7f93bc0a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d0103b60 0x7f93d019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93d01031a0 0x7f93d0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 --2- 172.21.15.37:0/536288932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93d0101fb0 0x7f93d01981e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 >> 172.21.15.37:0/536288932 conn(0x7f93d00fd520 msgr2=0x7f93d0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 shutdown_connections 2024-04-23T18:18:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.459+0000 7f93d8205640 1 -- 172.21.15.37:0/536288932 wait complete. 2024-04-23T18:18:40.463 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 -- 172.21.15.37:0/3917585691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 msgr2=0x7f87d41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 --2- 172.21.15.37:0/3917585691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d41023c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402a410 comp rx=0 tx=0).stop 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 -- 172.21.15.37:0/3917585691 shutdown_connections 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 --2- 172.21.15.37:0/3917585691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103b50 0x7f87d410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 --2- 172.21.15.37:0/3917585691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87d4103190 0x7f87d4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 --2- 172.21.15.37:0/3917585691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d41023c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 -- 172.21.15.37:0/3917585691 >> 172.21.15.37:0/3917585691 conn(0x7f87d40fd530 msgr2=0x7f87d40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 -- 172.21.15.37:0/3917585691 shutdown_connections 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.577+0000 7f87d8944640 1 -- 172.21.15.37:0/3917585691 wait complete. 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 Processor -- start 2024-04-23T18:18:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 -- start start 2024-04-23T18:18:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103190 0x7f87d4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87d4103b50 0x7f87d419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87d410dae0 con 0x7f87d4101fa0 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87d410d960 con 0x7f87d4103190 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d8944640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f87d410dc60 con 0x7f87d4103b50 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43100/0 (socket says 172.21.15.37:43100) 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d259c640 1 -- 172.21.15.37:0/541779200 learned_addr learned my addr 172.21.15.37:0/541779200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87c9d9b640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103190 0x7f87d4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.578+0000 7f87d2d9d640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87d4103b50 0x7f87d419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d259c640 1 -- 172.21.15.37:0/541779200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87d4103b50 msgr2=0x7f87d419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d259c640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87d4103b50 0x7f87d419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d259c640 1 -- 172.21.15.37:0/541779200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103190 msgr2=0x7f87d4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d259c640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103190 0x7f87d4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d259c640 1 -- 172.21.15.37:0/541779200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d419fe50 con 0x7f87d4101fa0 2024-04-23T18:18:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87c9d9b640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103190 0x7f87d4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d259c640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d4198230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006010 tx=0x7f87c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c403f030 con 0x7f87d4101fa0 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c4031910 con 0x7f87d4101fa0 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d41a00e0 con 0x7f87d4101fa0 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4035070 con 0x7f87d4101fa0 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.579+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d41a05d0 con 0x7f87d4101fa0 2024-04-23T18:18:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.580+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d4109480 con 0x7f87d4101fa0 2024-04-23T18:18:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.581+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f87c4042030 con 0x7f87d4101fa0 2024-04-23T18:18:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.582+0000 7f87cbfff640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f87c00a3bb0 0x7f87c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.582+0000 7f87c9d9b640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f87c00a3bb0 0x7f87c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.582+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f87c40f0690 con 0x7f87d4101fa0 2024-04-23T18:18:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.582+0000 7f87c9d9b640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f87c00a3bb0 0x7f87c00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87d4199740 tx=0x7f87bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.586+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87c40bd1c0 con 0x7f87d4101fa0 2024-04-23T18:18:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.799+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f87d4062420 con 0x7f87c00a3bb0 2024-04-23T18:18:40.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.823+0000 7f87cbfff640 1 -- 172.21.15.37:0/541779200 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f87d4062420 con 0x7f87c00a3bb0 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f87c00a3bb0 msgr2=0x7f87c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f87c00a3bb0 0x7f87c00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87d4199740 tx=0x7f87bc005f90 comp rx=0 tx=0).stop 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 msgr2=0x7f87d4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d4198230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006010 tx=0x7f87c40042f0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 shutdown_connections 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f87c00a3bb0 0x7f87c00a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87d4103b50 0x7f87d419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87d4103190 0x7f87d4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 --2- 172.21.15.37:0/541779200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87d4101fa0 0x7f87d4198230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 >> 172.21.15.37:0/541779200 conn(0x7f87d40fd530 msgr2=0x7f87d4106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:40.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 shutdown_connections 2024-04-23T18:18:40.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.826+0000 7f87d8944640 1 -- 172.21.15.37:0/541779200 wait complete. 2024-04-23T18:18:40.829 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:18:40.436+0000 7f828a8ba700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:18:40.829 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:18:40.436+0000 7f828a8ba700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:18:40.829 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:18:40.436+0000 7f828a8ba700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:18:40.829 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:18:40.436+0000 7f828a8ba700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:18:40.829 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:18:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:18:40.436+0000 7f828a8ba700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:18:40.830 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.944+0000 7f2060ca7640 1 -- 172.21.15.37:0/727696723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c100e90 msgr2=0x7f205c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.944+0000 7f2060ca7640 1 --2- 172.21.15.37:0/727696723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c100e90 0x7f205c1012b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2054009900 tx=0x7f205402a410 comp rx=0 tx=0).stop 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- 172.21.15.37:0/727696723 shutdown_connections 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 --2- 172.21.15.37:0/727696723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c1029a0 0x7f205c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 --2- 172.21.15.37:0/727696723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c101fe0 0x7f205c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 --2- 172.21.15.37:0/727696723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c100e90 0x7f205c1012b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- 172.21.15.37:0/727696723 >> 172.21.15.37:0/727696723 conn(0x7f205c0fc400 msgr2=0x7f205c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- 172.21.15.37:0/727696723 shutdown_connections 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- 172.21.15.37:0/727696723 wait complete. 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 Processor -- start 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- start start 2024-04-23T18:18:40.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c100e90 0x7f205c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c101fe0 0x7f205c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 0x7f205c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10c9e0 con 0x7f205c1029a0 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10c860 con 0x7f205c100e90 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.945+0000 7f2060ca7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10cb60 con 0x7f205c101fe0 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 0x7f205c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 0x7f205c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43114/0 (socket says 172.21.15.37:43114) 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 -- 172.21.15.37:0/1236669016 learned_addr learned my addr 172.21.15.37:0/1236669016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205a59c640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c100e90 0x7f205c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f2059d9b640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c101fe0 0x7f205c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 -- 172.21.15.37:0/1236669016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c101fe0 msgr2=0x7f205c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c101fe0 0x7f205c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 -- 172.21.15.37:0/1236669016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c100e90 msgr2=0x7f205c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c100e90 0x7f205c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:40.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 -- 172.21.15.37:0/1236669016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c19fe90 con 0x7f205c1029a0 2024-04-23T18:18:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f205ad9d640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 0x7f205c19f820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f204c00eb10 tx=0x7f204c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c004b20 con 0x7f205c1029a0 2024-04-23T18:18:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f204c004cc0 con 0x7f205c1029a0 2024-04-23T18:18:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c010670 con 0x7f205c1029a0 2024-04-23T18:18:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1a0180 con 0x7f205c1029a0 2024-04-23T18:18:40.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.946+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c1a0680 con 0x7f205c1029a0 2024-04-23T18:18:40.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.947+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c1a0940 con 0x7f205c1029a0 2024-04-23T18:18:40.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.948+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f204c005020 con 0x7f205c1029a0 2024-04-23T18:18:40.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.949+0000 7f204b7fe640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20440a3bb0 0x7f20440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:40.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.949+0000 7f205a59c640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20440a3bb0 0x7f20440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:40.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.949+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f204c0d0900 con 0x7f205c1029a0 2024-04-23T18:18:40.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.950+0000 7f205a59c640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20440a3bb0 0x7f20440a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2054000c00 tx=0x7f20540314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:40.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:40.954+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f204c09d390 con 0x7f205c1029a0 2024-04-23T18:18:41.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:41 smithi037 ceph-mon[29404]: pgmap v288: 97 pgs: 92 active+clean, 5 creating+peering; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 12 op/s 2024-04-23T18:18:41.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:41 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4223492374' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:18:41.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:41 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] up:active 2024-04-23T18:18:41.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:41 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-23T18:18:41.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:41 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:18:41.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:41 smithi037 ceph-mon[29404]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:41.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.175+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f205c0698a0 con 0x7f20440a3bb0 2024-04-23T18:18:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.181+0000 7f204b7fe640 1 -- 172.21.15.37:0/1236669016 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f205c0698a0 con 0x7f20440a3bb0 2024-04-23T18:18:41.174 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/4993588a-d1c2-4268-b07c-ee51c59abfb2 2024-04-23T18:18:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20440a3bb0 msgr2=0x7f20440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20440a3bb0 0x7f20440a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2054000c00 tx=0x7f20540314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 msgr2=0x7f205c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 0x7f205c19f820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f204c00eb10 tx=0x7f204c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 shutdown_connections 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20440a3bb0 0x7f20440a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f205c1029a0 0x7f205c19f820 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c101fe0 0x7f205c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 --2- 172.21.15.37:0/1236669016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f205c100e90 0x7f205c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 >> 172.21.15.37:0/1236669016 conn(0x7f205c0fc400 msgr2=0x7f205c069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 shutdown_connections 2024-04-23T18:18:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.183+0000 7f2060ca7640 1 -- 172.21.15.37:0/1236669016 wait complete. 2024-04-23T18:18:41.187 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-23T18:18:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 -- 172.21.15.37:0/3533564546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938101fa0 msgr2=0x7fb9381023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 --2- 172.21.15.37:0/3533564546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938101fa0 0x7fb9381023c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb928009900 tx=0x7fb92802a410 comp rx=0 tx=0).stop 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 -- 172.21.15.37:0/3533564546 shutdown_connections 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 --2- 172.21.15.37:0/3533564546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb938103b50 0x7fb93810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 --2- 172.21.15.37:0/3533564546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb938103190 0x7fb938103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 --2- 172.21.15.37:0/3533564546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938101fa0 0x7fb9381023c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 -- 172.21.15.37:0/3533564546 >> 172.21.15.37:0/3533564546 conn(0x7fb9380fd530 msgr2=0x7fb9380ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 -- 172.21.15.37:0/3533564546 shutdown_connections 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 -- 172.21.15.37:0/3533564546 wait complete. 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 Processor -- start 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.301+0000 7fb93e059640 1 -- start start 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb938101fa0 0x7fb938198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb938103190 0x7fb938198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 0x7fb93819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810dae0 con 0x7fb938103b50 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810d960 con 0x7fb938101fa0 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810dc60 con 0x7fb938103190 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 0x7fb93819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 0x7fb93819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43124/0 (socket says 172.21.15.37:43124) 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 -- 172.21.15.37:0/4172982670 learned_addr learned my addr 172.21.15.37:0/4172982670 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb9377fe640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb938101fa0 0x7fb938198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb92effd640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb938103190 0x7fb938198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 -- 172.21.15.37:0/4172982670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb938103190 msgr2=0x7fb938198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb938103190 0x7fb938198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 -- 172.21.15.37:0/4172982670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb938101fa0 msgr2=0x7fb938198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb938101fa0 0x7fb938198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 -- 172.21.15.37:0/4172982670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93819fe20 con 0x7fb938103b50 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb937fff640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 0x7fb93819f7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb93000eaa0 tx=0x7fb93000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93000cdb0 con 0x7fb938103b50 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb930005020 con 0x7fb938103b50 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9381a0110 con 0x7fb938103b50 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300116d0 con 0x7fb938103b50 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.302+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9381a06b0 con 0x7fb938103b50 2024-04-23T18:18:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.303+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9381093e0 con 0x7fb938103b50 2024-04-23T18:18:41.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.305+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9300051d0 con 0x7fb938103b50 2024-04-23T18:18:41.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.305+0000 7fb9357fa640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9240a3c30 0x7fb9240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.305+0000 7fb9377fe640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9240a3c30 0x7fb9240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.305+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb9300d0420 con 0x7fb938103b50 2024-04-23T18:18:41.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.306+0000 7fb9377fe640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9240a3c30 0x7fb9240a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb928009230 tx=0x7fb9280314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:41.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.310+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb930016030 con 0x7fb938103b50 2024-04-23T18:18:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:41 smithi172 ceph-mon[32456]: pgmap v288: 97 pgs: 92 active+clean, 5 creating+peering; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 12 op/s 2024-04-23T18:18:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:41 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4223492374' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:18:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:41 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] up:active 2024-04-23T18:18:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:41 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-23T18:18:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:41 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:18:41.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:41 smithi172 ceph-mon[32456]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:41 smithi173 ceph-mon[32906]: pgmap v288: 97 pgs: 92 active+clean, 5 creating+peering; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 12 op/s 2024-04-23T18:18:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:41 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4223492374' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:18:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:41 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] up:active 2024-04-23T18:18:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:41 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-23T18:18:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:41 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:18:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:41 smithi173 ceph-mon[32906]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.523+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb938062390 con 0x7fb9240a3c30 2024-04-23T18:18:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.550+0000 7fb9357fa640 1 -- 172.21.15.37:0/4172982670 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb938062390 con 0x7fb9240a3c30 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9240a3c30 msgr2=0x7fb9240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9240a3c30 0x7fb9240a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb928009230 tx=0x7fb9280314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 msgr2=0x7fb93819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 0x7fb93819f7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb93000eaa0 tx=0x7fb93000ef60 comp rx=0 tx=0).stop 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 shutdown_connections 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9240a3c30 0x7fb9240a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb938103b50 0x7fb93819f7b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb938103190 0x7fb938198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 --2- 172.21.15.37:0/4172982670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb938101fa0 0x7fb938198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 >> 172.21.15.37:0/4172982670 conn(0x7fb9380fd530 msgr2=0x7fb9381065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:41.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 shutdown_connections 2024-04-23T18:18:41.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.552+0000 7fb93e059640 1 -- 172.21.15.37:0/4172982670 wait complete. 2024-04-23T18:18:41.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-23T18:18:41.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 -- 172.21.15.37:0/4221536690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58075f10 msgr2=0x7fbf58074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 --2- 172.21.15.37:0/4221536690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58075f10 0x7fbf58074360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf5402a480 comp rx=0 tx=0).stop 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 -- 172.21.15.37:0/4221536690 shutdown_connections 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 --2- 172.21.15.37:0/4221536690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf58109e60 0x7fbf5810c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 --2- 172.21.15.37:0/4221536690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf58074930 0x7fbf58074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 --2- 172.21.15.37:0/4221536690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58075f10 0x7fbf58074360 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.670+0000 7fbf6007c640 1 -- 172.21.15.37:0/4221536690 >> 172.21.15.37:0/4221536690 conn(0x7fbf580fd520 msgr2=0x7fbf580ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 -- 172.21.15.37:0/4221536690 shutdown_connections 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 -- 172.21.15.37:0/4221536690 wait complete. 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 Processor -- start 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 -- start start 2024-04-23T18:18:41.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf58074930 0x7fbf5819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf58075f10 0x7fbf5819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 0x7fbf581a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810fd00 con 0x7fbf58109e60 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810fb80 con 0x7fbf58074930 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf6007c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5810fe80 con 0x7fbf58075f10 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf5e619640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 0x7fbf581a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf5e619640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 0x7fbf581a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43136/0 (socket says 172.21.15.37:43136) 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf5e619640 1 -- 172.21.15.37:0/1157038725 learned_addr learned my addr 172.21.15.37:0/1157038725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.671+0000 7fbf5de18640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf58074930 0x7fbf5819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5d617640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf58075f10 0x7fbf5819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5e619640 1 -- 172.21.15.37:0/1157038725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf58075f10 msgr2=0x7fbf5819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5e619640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf58075f10 0x7fbf5819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5e619640 1 -- 172.21.15.37:0/1157038725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf58074930 msgr2=0x7fbf5819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5e619640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf58074930 0x7fbf5819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5e619640 1 -- 172.21.15.37:0/1157038725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf581a2050 con 0x7fbf58109e60 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf5e619640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 0x7fbf581a19e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009fd0 tx=0x7fbf5000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50017070 con 0x7fbf58109e60 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50003e20 con 0x7fbf58109e60 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581a2340 con 0x7fbf58109e60 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50010460 con 0x7fbf58109e60 2024-04-23T18:18:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.672+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a2890 con 0x7fbf58109e60 2024-04-23T18:18:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.673+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58108c20 con 0x7fbf58109e60 2024-04-23T18:18:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.674+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf50003970 con 0x7fbf58109e60 2024-04-23T18:18:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.675+0000 7fbf4effd640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbf440a3c30 0x7fbf440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.675+0000 7fbf5de18640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbf440a3c30 0x7fbf440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.675+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fbf500d0090 con 0x7fbf58109e60 2024-04-23T18:18:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.675+0000 7fbf5de18640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbf440a3c30 0x7fbf440a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000c00 tx=0x7fbf540314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.679+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf5009cb20 con 0x7fbf58109e60 2024-04-23T18:18:41.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.895+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbf58062420 con 0x7fbf440a3c30 2024-04-23T18:18:41.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.900+0000 7fbf4effd640 1 -- 172.21.15.37:0/1157038725 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbf58062420 con 0x7fbf440a3c30 2024-04-23T18:18:41.893 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/d3b18bbc-ac32-4ec7-a1cb-5a6ae034703b 2024-04-23T18:18:41.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbf440a3c30 msgr2=0x7fbf440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbf440a3c30 0x7fbf440a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000c00 tx=0x7fbf540314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 msgr2=0x7fbf581a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 0x7fbf581a19e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009fd0 tx=0x7fbf5000b260 comp rx=0 tx=0).stop 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 shutdown_connections 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbf440a3c30 0x7fbf440a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf58109e60 0x7fbf581a19e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf58075f10 0x7fbf5819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 --2- 172.21.15.37:0/1157038725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf58074930 0x7fbf5819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 >> 172.21.15.37:0/1157038725 conn(0x7fbf580fd520 msgr2=0x7fbf58105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 shutdown_connections 2024-04-23T18:18:41.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:41.902+0000 7fbf6007c640 1 -- 172.21.15.37:0/1157038725 wait complete. 2024-04-23T18:18:41.906 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:18:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 -- 172.21.15.37:0/3633159218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 msgr2=0x7f5454104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 --2- 172.21.15.37:0/3633159218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f5454104e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5450009900 tx=0x7f545002a410 comp rx=0 tx=0).stop 2024-04-23T18:18:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 -- 172.21.15.37:0/3633159218 shutdown_connections 2024-04-23T18:18:42.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 --2- 172.21.15.37:0/3633159218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 0x7f545410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 --2- 172.21.15.37:0/3633159218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5454105360 0x7f5454107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 --2- 172.21.15.37:0/3633159218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f5454104e20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 -- 172.21.15.37:0/3633159218 >> 172.21.15.37:0/3633159218 conn(0x7f54540fc440 msgr2=0x7f54540fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 -- 172.21.15.37:0/3633159218 shutdown_connections 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.014+0000 7f545c6cf640 1 -- 172.21.15.37:0/3633159218 wait complete. 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 Processor -- start 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 -- start start 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f54541982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5454105360 0x7f54541987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 0x7f545419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f545410dba0 con 0x7f5454102a30 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f545410da20 con 0x7f5454107c90 2024-04-23T18:18:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545c6cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f545410dd20 con 0x7f5454105360 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f54541982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f54541982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43150/0 (socket says 172.21.15.37:43150) 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 -- 172.21.15.37:0/838118149 learned_addr learned my addr 172.21.15.37:0/838118149 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 -- 172.21.15.37:0/838118149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5454105360 msgr2=0x7f54541987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f5459c6a640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5454105360 0x7f54541987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545ac6c640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 0x7f545419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5454105360 0x7f54541987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 -- 172.21.15.37:0/838118149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 msgr2=0x7f545419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 0x7f545419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 -- 172.21.15.37:0/838118149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545419fe50 con 0x7f5454102a30 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545ac6c640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 0x7f545419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.015+0000 7f545a46b640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f54541982a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5450031500 tx=0x7f5450003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:42.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.016+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545003f070 con 0x7f5454102a30 2024-04-23T18:18:42.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.016+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545003b460 con 0x7f5454102a30 2024-04-23T18:18:42.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.016+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450035030 con 0x7f5454102a30 2024-04-23T18:18:42.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.016+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54541a00e0 con 0x7f5454102a30 2024-04-23T18:18:42.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.016+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541a06a0 con 0x7f5454102a30 2024-04-23T18:18:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.017+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54540fdf30 con 0x7f5454102a30 2024-04-23T18:18:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.018+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5450042070 con 0x7f5454102a30 2024-04-23T18:18:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.018+0000 7f544b7fe640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f54400a3c30 0x7f54400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.018+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f54500f1130 con 0x7f5454102a30 2024-04-23T18:18:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.018+0000 7f5459c6a640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f54400a3c30 0x7f54400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.019+0000 7f5459c6a640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f54400a3c30 0x7f54400a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5454199740 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.022+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54500bdc60 con 0x7f5454102a30 2024-04-23T18:18:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.097+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f54500b9bd0 con 0x7f5454102a30 2024-04-23T18:18:42.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:42 smithi037 ceph-mon[29404]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:42 smithi037 ceph-mon[29404]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:42 smithi037 ceph-mon[29404]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.255+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f545402d5c0 con 0x7f5454102a30 2024-04-23T18:18:42.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.256+0000 7f544b7fe640 1 -- 172.21.15.37:0/838118149 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f54500dfa20 con 0x7f5454102a30 2024-04-23T18:18:42.249 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:42.249 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:18:40.084107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:18:42.249 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:18:42.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f54400a3c30 msgr2=0x7f54400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f54400a3c30 0x7f54400a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5454199740 tx=0x7f5444008040 comp rx=0 tx=0).stop 2024-04-23T18:18:42.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 msgr2=0x7f54541982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f54541982a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5450031500 tx=0x7f5450003d50 comp rx=0 tx=0).stop 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 shutdown_connections 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f54400a3c30 0x7f54400a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5454107c90 0x7f545419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5454105360 0x7f54541987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 --2- 172.21.15.37:0/838118149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5454102a30 0x7f54541982a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 >> 172.21.15.37:0/838118149 conn(0x7f54540fc440 msgr2=0x7f54541033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 shutdown_connections 2024-04-23T18:18:42.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.258+0000 7f545c6cf640 1 -- 172.21.15.37:0/838118149 wait complete. 2024-04-23T18:18:42.263 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:18:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:42 smithi172 ceph-mon[32456]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:42 smithi172 ceph-mon[32456]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:42 smithi172 ceph-mon[32456]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:42 smithi173 ceph-mon[32906]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:42 smithi173 ceph-mon[32906]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:42 smithi173 ceph-mon[32906]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:42.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 -- 172.21.15.37:0/1859367493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18103190 msgr2=0x7f3d18103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1859367493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18103190 0x7f3d18103610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009900 tx=0x7f3d0802a410 comp rx=0 tx=0).stop 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 -- 172.21.15.37:0/1859367493 shutdown_connections 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1859367493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d18103b50 0x7f3d1810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1859367493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18103190 0x7f3d18103610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1859367493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18101fa0 0x7f3d181023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 -- 172.21.15.37:0/1859367493 >> 172.21.15.37:0/1859367493 conn(0x7f3d180fd530 msgr2=0x7f3d180ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 -- 172.21.15.37:0/1859367493 shutdown_connections 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.373+0000 7f3d1f060640 1 -- 172.21.15.37:0/1859367493 wait complete. 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 Processor -- start 2024-04-23T18:18:42.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 -- start start 2024-04-23T18:18:42.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 0x7f3d18075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d18103190 0x7f3d18074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18103b50 0x7f3d18074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1810dae0 con 0x7f3d18101fa0 2024-04-23T18:18:42.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1810d960 con 0x7f3d18103190 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1f060640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1810dc60 con 0x7f3d18103b50 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 0x7f3d18075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 0x7f3d18075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43168/0 (socket says 172.21.15.37:43168) 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 -- 172.21.15.37:0/1373165596 learned_addr learned my addr 172.21.15.37:0/1373165596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d0ffff640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d18103190 0x7f3d18074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1d5fd640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18103b50 0x7f3d18074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 -- 172.21.15.37:0/1373165596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18103b50 msgr2=0x7f3d18074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18103b50 0x7f3d18074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 -- 172.21.15.37:0/1373165596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d18103190 msgr2=0x7f3d18074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d18103190 0x7f3d18074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 -- 172.21.15.37:0/1373165596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d18075110 con 0x7f3d18101fa0 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1d5fd640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18103b50 0x7f3d18074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.374+0000 7f3d1cdfc640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 0x7f3d18075d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400d950 tx=0x7f3d1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:42.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.375+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14017020 con 0x7f3d18101fa0 2024-04-23T18:18:42.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.375+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d181a22f0 con 0x7f3d18101fa0 2024-04-23T18:18:42.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.375+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d14009c30 con 0x7f3d18101fa0 2024-04-23T18:18:42.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.375+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400f6b0 con 0x7f3d18101fa0 2024-04-23T18:18:42.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.375+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d181a2890 con 0x7f3d18101fa0 2024-04-23T18:18:42.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.376+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d18109340 con 0x7f3d18101fa0 2024-04-23T18:18:42.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.377+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d1401a020 con 0x7f3d18101fa0 2024-04-23T18:18:42.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.377+0000 7f3d0dffb640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3d000a3c40 0x7f3d000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.377+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3d140d0190 con 0x7f3d18101fa0 2024-04-23T18:18:42.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.377+0000 7f3d0ffff640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3d000a3c40 0x7f3d000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.378+0000 7f3d0ffff640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3d000a3c40 0x7f3d000a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d18075a70 tx=0x7f3d080314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:42.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.382+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d14009de0 con 0x7f3d18101fa0 2024-04-23T18:18:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.569+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d18062b50 con 0x7f3d18101fa0 2024-04-23T18:18:42.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.571+0000 7f3d0dffb640 1 -- 172.21.15.37:0/1373165596 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22243 (secure 0 0 0) 0x7f3d1409cbc0 con 0x7f3d18101fa0 2024-04-23T18:18:42.565 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:42.565 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:18:36.045489+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:18:42.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3d000a3c40 msgr2=0x7f3d000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3d000a3c40 0x7f3d000a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d18075a70 tx=0x7f3d080314a0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 msgr2=0x7f3d18075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 0x7f3d18075d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400d950 tx=0x7f3d1400de10 comp rx=0 tx=0).stop 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 shutdown_connections 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3d000a3c40 0x7f3d000a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d18103b50 0x7f3d18074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d18103190 0x7f3d18074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 --2- 172.21.15.37:0/1373165596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d18101fa0 0x7f3d18075d00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 >> 172.21.15.37:0/1373165596 conn(0x7f3d180fd530 msgr2=0x7f3d18106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 shutdown_connections 2024-04-23T18:18:42.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.573+0000 7f3d1f060640 1 -- 172.21.15.37:0/1373165596 wait complete. 2024-04-23T18:18:42.578 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T18:18:42.578 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:18:42.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.686+0000 7f5caf44e640 1 -- 172.21.15.37:0/619855750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8102a10 msgr2=0x7f5ca8104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.686+0000 7f5caf44e640 1 --2- 172.21.15.37:0/619855750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8102a10 0x7f5ca8104e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009ca0 tx=0x7f5c9802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.686+0000 7f5caf44e640 1 -- 172.21.15.37:0/619855750 shutdown_connections 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 --2- 172.21.15.37:0/619855750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8107c70 0x7f5ca810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 --2- 172.21.15.37:0/619855750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8105340 0x7f5ca8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 --2- 172.21.15.37:0/619855750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8102a10 0x7f5ca8104e00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- 172.21.15.37:0/619855750 >> 172.21.15.37:0/619855750 conn(0x7f5ca80fc400 msgr2=0x7f5ca80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- 172.21.15.37:0/619855750 shutdown_connections 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- 172.21.15.37:0/619855750 wait complete. 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 Processor -- start 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- start start 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8102a10 0x7f5ca81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 0x7f5ca8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8107c70 0x7f5ca819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca810dad0 con 0x7f5ca8105340 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca810d950 con 0x7f5ca8107c70 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5caf44e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca810dc50 con 0x7f5ca8102a10 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5cac9e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 0x7f5ca8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.687+0000 7f5cac9e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 0x7f5ca8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43190/0 (socket says 172.21.15.37:43190) 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 -- 172.21.15.37:0/294507509 learned_addr learned my addr 172.21.15.37:0/294507509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cad9eb640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8107c70 0x7f5ca819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cad1ea640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8102a10 0x7f5ca81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 -- 172.21.15.37:0/294507509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8102a10 msgr2=0x7f5ca81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8102a10 0x7f5ca81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 -- 172.21.15.37:0/294507509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8107c70 msgr2=0x7f5ca819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8107c70 0x7f5ca819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 -- 172.21.15.37:0/294507509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca819fdf0 con 0x7f5ca8105340 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cad1ea640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8102a10 0x7f5ca81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5cac9e9640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 0x7f5ca8198710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00d740 tx=0x7f5c9c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c017070 con 0x7f5ca8105340 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c9c00de50 con 0x7f5ca8105340 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca81a00e0 con 0x7f5ca8105340 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c004ce0 con 0x7f5ca8105340 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.688+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca81a05e0 con 0x7f5ca8105340 2024-04-23T18:18:42.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.689+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca80fdef0 con 0x7f5ca8105340 2024-04-23T18:18:42.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.690+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5c9c004e80 con 0x7f5ca8105340 2024-04-23T18:18:42.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.691+0000 7f5ca67fc640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c900a3c40 0x7f5c900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:42.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.691+0000 7f5cad1ea640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c900a3c40 0x7f5c900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:42.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.691+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5c9c0cfe80 con 0x7f5ca8105340 2024-04-23T18:18:42.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.692+0000 7f5cad1ea640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c900a3c40 0x7f5c900a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c98031b50 tx=0x7f5c98005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:42.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.696+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c9c09c7b0 con 0x7f5ca8105340 2024-04-23T18:18:42.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.878+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ca8062390 con 0x7f5c900a3c40 2024-04-23T18:18:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.886+0000 7f5ca67fc640 1 -- 172.21.15.37:0/294507509 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258759 (secure 0 0 0) 0x7f5ca8062390 con 0x7f5c900a3c40 2024-04-23T18:18:42.882 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c900a3c40 msgr2=0x7f5c900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c900a3c40 0x7f5c900a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c98031b50 tx=0x7f5c98005ef0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 msgr2=0x7f5ca8198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 0x7f5ca8198710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00d740 tx=0x7f5c9c00dc00 comp rx=0 tx=0).stop 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 shutdown_connections 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c900a3c40 0x7f5c900a6100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8107c70 0x7f5ca819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ca8105340 0x7f5ca8198710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 --2- 172.21.15.37:0/294507509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ca8102a10 0x7f5ca81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:42.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 >> 172.21.15.37:0/294507509 conn(0x7f5ca80fc400 msgr2=0x7f5ca8103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 shutdown_connections 2024-04-23T18:18:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:42.889+0000 7f5caf44e640 1 -- 172.21.15.37:0/294507509 wait complete. 2024-04-23T18:18:42.893 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-23T18:18:41.861077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595869,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":146,"num_read_kb":128,"num_write":299,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374724,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567100,"statfs":{"total":1151940427776,"available":1151556710400,"internally_reserved":0,"allocated":13389824,"data_stored":7262985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5501,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":118,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.828725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054481+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:18:36.054481+0000","last_peered":"2024-04-23T18:18:36.054481+0000","last_clean":"2024-04-23T18:18:36.054481+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:18:36.054481+0000","last_undegraded":"2024-04-23T18:18:36.054481+0000","last_fullsized":"2024-04-23T18:18:36.054481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607148+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:18:36.607148+0000","last_peered":"2024-04-23T18:18:36.607148+0000","last_clean":"2024-04-23T18:18:36.607148+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:18:36.607148+0000","last_undegraded":"2024-04-23T18:18:36.607148+0000","last_fullsized":"2024-04-23T18:18:36.607148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607062+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:18:36.607062+0000","last_peered":"2024-04-23T18:18:36.607062+0000","last_clean":"2024-04-23T18:18:36.607062+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:18:36.607062+0000","last_undegraded":"2024-04-23T18:18:36.607062+0000","last_fullsized":"2024-04-23T18:18:36.607062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606681+0000","last_change":"2024-04-23T18:18:35.065476+0000","last_active":"2024-04-23T18:18:36.606681+0000","last_peered":"2024-04-23T18:18:36.606681+0000","last_clean":"2024-04-23T18:18:36.606681+0000","last_became_active":"2024-04-23T18:18:35.065075+0000","last_became_peered":"2024-04-23T18:18:35.065075+0000","last_unstale":"2024-04-23T18:18:36.606681+0000","last_undegraded":"2024-04-23T18:18:36.606681+0000","last_fullsized":"2024-04-23T18:18:36.606681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:37.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606176+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:18:36.606176+0000","last_peered":"2024-04-23T18:18:36.606176+0000","last_clean":"2024-04-23T18:18:36.606176+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:18:36.606176+0000","last_undegraded":"2024-04-23T18:18:36.606176+0000","last_fullsized":"2024-04-23T18:18:36.606176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054348+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:18:36.054348+0000","last_peered":"2024-04-23T18:18:36.054348+0000","last_clean":"2024-04-23T18:18:36.054348+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:18:36.054348+0000","last_undegraded":"2024-04-23T18:18:36.054348+0000","last_fullsized":"2024-04-23T18:18:36.054348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997584+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:18:36.997584+0000","last_peered":"2024-04-23T18:18:36.997584+0000","last_clean":"2024-04-23T18:18:36.997584+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:18:36.997584+0000","last_undegraded":"2024-04-23T18:18:36.997584+0000","last_fullsized":"2024-04-23T18:18:36.997584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606246+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:18:36.606246+0000","last_peered":"2024-04-23T18:18:36.606246+0000","last_clean":"2024-04-23T18:18:36.606246+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:18:36.606246+0000","last_undegraded":"2024-04-23T18:18:36.606246+0000","last_fullsized":"2024-04-23T18:18:36.606246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606742+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:18:36.606742+0000","last_peered":"2024-04-23T18:18:36.606742+0000","last_clean":"2024-04-23T18:18:36.606742+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:18:36.606742+0000","last_undegraded":"2024-04-23T18:18:36.606742+0000","last_fullsized":"2024-04-23T18:18:36.606742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054346+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:18:36.054346+0000","last_peered":"2024-04-23T18:18:36.054346+0000","last_clean":"2024-04-23T18:18:36.054346+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:18:36.054346+0000","last_undegraded":"2024-04-23T18:18:36.054346+0000","last_fullsized":"2024-04-23T18:18:36.054346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997899+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:18:36.997899+0000","last_peered":"2024-04-23T18:18:36.997899+0000","last_clean":"2024-04-23T18:18:36.997899+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:18:36.997899+0000","last_undegraded":"2024-04-23T18:18:36.997899+0000","last_fullsized":"2024-04-23T18:18:36.997899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606671+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:18:36.606671+0000","last_peered":"2024-04-23T18:18:36.606671+0000","last_clean":"2024-04-23T18:18:36.606671+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:18:36.606671+0000","last_undegraded":"2024-04-23T18:18:36.606671+0000","last_fullsized":"2024-04-23T18:18:36.606671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607484+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:18:36.607484+0000","last_peered":"2024-04-23T18:18:36.607484+0000","last_clean":"2024-04-23T18:18:36.607484+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:18:36.607484+0000","last_undegraded":"2024-04-23T18:18:36.607484+0000","last_fullsized":"2024-04-23T18:18:36.607484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606827+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:18:36.606827+0000","last_peered":"2024-04-23T18:18:36.606827+0000","last_clean":"2024-04-23T18:18:36.606827+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:18:36.606827+0000","last_undegraded":"2024-04-23T18:18:36.606827+0000","last_fullsized":"2024-04-23T18:18:36.606827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606769+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:18:36.606769+0000","last_peered":"2024-04-23T18:18:36.606769+0000","last_clean":"2024-04-23T18:18:36.606769+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:18:36.606769+0000","last_undegraded":"2024-04-23T18:18:36.606769+0000","last_fullsized":"2024-04-23T18:18:36.606769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606089+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:18:36.606089+0000","last_peered":"2024-04-23T18:18:36.606089+0000","last_clean":"2024-04-23T18:18:36.606089+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:18:36.606089+0000","last_undegraded":"2024-04-23T18:18:36.606089+0000","last_fullsized":"2024-04-23T18:18:36.606089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054598+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:18:36.054598+0000","last_peered":"2024-04-23T18:18:36.054598+0000","last_clean":"2024-04-23T18:18:36.054598+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:18:36.054598+0000","last_undegraded":"2024-04-23T18:18:36.054598+0000","last_fullsized":"2024-04-23T18:18:36.054598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.053508+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:18:36.053508+0000","last_peered":"2024-04-23T18:18:36.053508+0000","last_clean":"2024-04-23T18:18:36.053508+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:18:36.053508+0000","last_undegraded":"2024-04-23T18:18:36.053508+0000","last_fullsized":"2024-04-23T18:18:36.053508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289295+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:18:36.289295+0000","last_peered":"2024-04-23T18:18:36.289295+0000","last_clean":"2024-04-23T18:18:36.289295+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:18:36.289295+0000","last_undegraded":"2024-04-23T18:18:36.289295+0000","last_fullsized":"2024-04-23T18:18:36.289295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.087161+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:18:39.087161+0000","last_peered":"2024-04-23T18:18:39.087161+0000","last_clean":"2024-04-23T18:18:39.087161+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:18:39.087161+0000","last_undegraded":"2024-04-23T18:18:39.087161+0000","last_fullsized":"2024-04-23T18:18:39.087161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607523+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:18:36.607523+0000","last_peered":"2024-04-23T18:18:36.607523+0000","last_clean":"2024-04-23T18:18:36.607523+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:18:36.607523+0000","last_undegraded":"2024-04-23T18:18:36.607523+0000","last_fullsized":"2024-04-23T18:18:36.607523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607793+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:18:36.607793+0000","last_peered":"2024-04-23T18:18:36.607793+0000","last_clean":"2024-04-23T18:18:36.607793+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:18:36.607793+0000","last_undegraded":"2024-04-23T18:18:36.607793+0000","last_fullsized":"2024-04-23T18:18:36.607793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608318+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:18:36.608318+0000","last_peered":"2024-04-23T18:18:36.608318+0000","last_clean":"2024-04-23T18:18:36.608318+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:18:36.608318+0000","last_undegraded":"2024-04-23T18:18:36.608318+0000","last_fullsized":"2024-04-23T18:18:36.608318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606537+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:18:36.606537+0000","last_peered":"2024-04-23T18:18:36.606537+0000","last_clean":"2024-04-23T18:18:36.606537+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:18:36.606537+0000","last_undegraded":"2024-04-23T18:18:36.606537+0000","last_fullsized":"2024-04-23T18:18:36.606537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606578+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:18:36.606578+0000","last_peered":"2024-04-23T18:18:36.606578+0000","last_clean":"2024-04-23T18:18:36.606578+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:18:36.606578+0000","last_undegraded":"2024-04-23T18:18:36.606578+0000","last_fullsized":"2024-04-23T18:18:36.606578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606432+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:18:36.606432+0000","last_peered":"2024-04-23T18:18:36.606432+0000","last_clean":"2024-04-23T18:18:36.606432+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:18:36.606432+0000","last_undegraded":"2024-04-23T18:18:36.606432+0000","last_fullsized":"2024-04-23T18:18:36.606432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.543061+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:18:36.543061+0000","last_peered":"2024-04-23T18:18:36.543061+0000","last_clean":"2024-04-23T18:18:36.543061+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:18:36.543061+0000","last_undegraded":"2024-04-23T18:18:36.543061+0000","last_fullsized":"2024-04-23T18:18:36.543061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.075812+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:18:38.075812+0000","last_peered":"2024-04-23T18:18:38.075812+0000","last_clean":"2024-04-23T18:18:38.075812+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:18:38.075812+0000","last_undegraded":"2024-04-23T18:18:38.075812+0000","last_fullsized":"2024-04-23T18:18:38.075812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606482+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:18:36.606482+0000","last_peered":"2024-04-23T18:18:36.606482+0000","last_clean":"2024-04-23T18:18:36.606482+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:18:36.606482+0000","last_undegraded":"2024-04-23T18:18:36.606482+0000","last_fullsized":"2024-04-23T18:18:36.606482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.083959+0000","last_change":"2024-04-23T18:18:34.049909+0000","last_active":"2024-04-23T18:18:39.083959+0000","last_peered":"2024-04-23T18:18:39.083959+0000","last_clean":"2024-04-23T18:18:39.083959+0000","last_became_active":"2024-04-23T18:18:34.049566+0000","last_became_peered":"2024-04-23T18:18:34.049566+0000","last_unstale":"2024-04-23T18:18:39.083959+0000","last_undegraded":"2024-04-23T18:18:39.083959+0000","last_fullsized":"2024-04-23T18:18:39.083959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:50:06.243216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.543108+0000","last_change":"2024-04-23T18:18:35.065394+0000","last_active":"2024-04-23T18:18:36.543108+0000","last_peered":"2024-04-23T18:18:36.543108+0000","last_clean":"2024-04-23T18:18:36.543108+0000","last_became_active":"2024-04-23T18:18:35.064723+0000","last_became_peered":"2024-04-23T18:18:35.064723+0000","last_unstale":"2024-04-23T18:18:36.543108+0000","last_undegraded":"2024-04-23T18:18:36.543108+0000","last_fullsized":"2024-04-23T18:18:36.543108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:21.430135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.085881+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:18:39.085881+0000","last_peered":"2024-04-23T18:18:39.085881+0000","last_clean":"2024-04-23T18:18:39.085881+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:18:39.085881+0000","last_undegraded":"2024-04-23T18:18:39.085881+0000","last_fullsized":"2024-04-23T18:18:39.085881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606250+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:18:36.606250+0000","last_peered":"2024-04-23T18:18:36.606250+0000","last_clean":"2024-04-23T18:18:36.606250+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:18:36.606250+0000","last_undegraded":"2024-04-23T18:18:36.606250+0000","last_fullsized":"2024-04-23T18:18:36.606250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542578+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:18:36.542578+0000","last_peered":"2024-04-23T18:18:36.542578+0000","last_clean":"2024-04-23T18:18:36.542578+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:18:36.542578+0000","last_undegraded":"2024-04-23T18:18:36.542578+0000","last_fullsized":"2024-04-23T18:18:36.542578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.053407+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:18:36.053407+0000","last_peered":"2024-04-23T18:18:36.053407+0000","last_clean":"2024-04-23T18:18:36.053407+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:18:36.053407+0000","last_undegraded":"2024-04-23T18:18:36.053407+0000","last_fullsized":"2024-04-23T18:18:36.053407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074246+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:18:38.074246+0000","last_peered":"2024-04-23T18:18:38.074246+0000","last_clean":"2024-04-23T18:18:38.074246+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:18:38.074246+0000","last_undegraded":"2024-04-23T18:18:38.074246+0000","last_fullsized":"2024-04-23T18:18:38.074246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606877+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:18:36.606877+0000","last_peered":"2024-04-23T18:18:36.606877+0000","last_clean":"2024-04-23T18:18:36.606877+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:18:36.606877+0000","last_undegraded":"2024-04-23T18:18:36.606877+0000","last_fullsized":"2024-04-23T18:18:36.606877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074905+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:18:38.074905+0000","last_peered":"2024-04-23T18:18:38.074905+0000","last_clean":"2024-04-23T18:18:38.074905+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:18:38.074905+0000","last_undegraded":"2024-04-23T18:18:38.074905+0000","last_fullsized":"2024-04-23T18:18:38.074905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608308+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:18:36.608308+0000","last_peered":"2024-04-23T18:18:36.608308+0000","last_clean":"2024-04-23T18:18:36.608308+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:18:36.608308+0000","last_undegraded":"2024-04-23T18:18:36.608308+0000","last_fullsized":"2024-04-23T18:18:36.608308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.543193+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:18:36.543193+0000","last_peered":"2024-04-23T18:18:36.543193+0000","last_clean":"2024-04-23T18:18:36.543193+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:18:36.543193+0000","last_undegraded":"2024-04-23T18:18:36.543193+0000","last_fullsized":"2024-04-23T18:18:36.543193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606901+0000","last_change":"2024-04-23T18:18:35.067537+0000","last_active":"2024-04-23T18:18:36.606901+0000","last_peered":"2024-04-23T18:18:36.606901+0000","last_clean":"2024-04-23T18:18:36.606901+0000","last_became_active":"2024-04-23T18:18:35.064318+0000","last_became_peered":"2024-04-23T18:18:35.064318+0000","last_unstale":"2024-04-23T18:18:36.606901+0000","last_undegraded":"2024-04-23T18:18:36.606901+0000","last_fullsized":"2024-04-23T18:18:36.606901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:35.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.073910+0000","last_change":"2024-04-23T18:18:34.048993+0000","last_active":"2024-04-23T18:18:38.073910+0000","last_peered":"2024-04-23T18:18:38.073910+0000","last_clean":"2024-04-23T18:18:38.073910+0000","last_became_active":"2024-04-23T18:18:34.048558+0000","last_became_peered":"2024-04-23T18:18:34.048558+0000","last_unstale":"2024-04-23T18:18:38.073910+0000","last_undegraded":"2024-04-23T18:18:38.073910+0000","last_fullsized":"2024-04-23T18:18:38.073910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:56:10.483326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606440+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:18:36.606440+0000","last_peered":"2024-04-23T18:18:36.606440+0000","last_clean":"2024-04-23T18:18:36.606440+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:18:36.606440+0000","last_undegraded":"2024-04-23T18:18:36.606440+0000","last_fullsized":"2024-04-23T18:18:36.606440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.086456+0000","last_change":"2024-04-23T18:18:34.047124+0000","last_active":"2024-04-23T18:18:39.086456+0000","last_peered":"2024-04-23T18:18:39.086456+0000","last_clean":"2024-04-23T18:18:39.086456+0000","last_became_active":"2024-04-23T18:18:34.046776+0000","last_became_peered":"2024-04-23T18:18:34.046776+0000","last_unstale":"2024-04-23T18:18:39.086456+0000","last_undegraded":"2024-04-23T18:18:39.086456+0000","last_fullsized":"2024-04-23T18:18:39.086456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:33.270309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606840+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:18:36.606840+0000","last_peered":"2024-04-23T18:18:36.606840+0000","last_clean":"2024-04-23T18:18:36.606840+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:18:36.606840+0000","last_undegraded":"2024-04-23T18:18:36.606840+0000","last_fullsized":"2024-04-23T18:18:36.606840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.075576+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:18:38.075576+0000","last_peered":"2024-04-23T18:18:38.075576+0000","last_clean":"2024-04-23T18:18:38.075576+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:18:38.075576+0000","last_undegraded":"2024-04-23T18:18:38.075576+0000","last_fullsized":"2024-04-23T18:18:38.075576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997770+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:18:36.997770+0000","last_peered":"2024-04-23T18:18:36.997770+0000","last_clean":"2024-04-23T18:18:36.997770+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:18:36.997770+0000","last_undegraded":"2024-04-23T18:18:36.997770+0000","last_fullsized":"2024-04-23T18:18:36.997770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:40.094404+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:18:40.094404+0000","last_peered":"2024-04-23T18:18:40.094404+0000","last_clean":"2024-04-23T18:18:40.094404+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:18:40.094404+0000","last_undegraded":"2024-04-23T18:18:40.094404+0000","last_fullsized":"2024-04-23T18:18:40.094404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997490+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:18:36.997490+0000","last_peered":"2024-04-23T18:18:36.997490+0000","last_clean":"2024-04-23T18:18:36.997490+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:18:36.997490+0000","last_undegraded":"2024-04-23T18:18:36.997490+0000","last_fullsized":"2024-04-23T18:18:36.997490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'10","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074438+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:18:38.074438+0000","last_peered":"2024-04-23T18:18:38.074438+0000","last_clean":"2024-04-23T18:18:38.074438+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:18:38.074438+0000","last_undegraded":"2024-04-23T18:18:38.074438+0000","last_fullsized":"2024-04-23T18:18:38.074438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542859+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:18:36.542859+0000","last_peered":"2024-04-23T18:18:36.542859+0000","last_clean":"2024-04-23T18:18:36.542859+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:18:36.542859+0000","last_undegraded":"2024-04-23T18:18:36.542859+0000","last_fullsized":"2024-04-23T18:18:36.542859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606412+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:18:36.606412+0000","last_peered":"2024-04-23T18:18:36.606412+0000","last_clean":"2024-04-23T18:18:36.606412+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:18:36.606412+0000","last_undegraded":"2024-04-23T18:18:36.606412+0000","last_fullsized":"2024-04-23T18:18:36.606412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054393+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:18:36.054393+0000","last_peered":"2024-04-23T18:18:36.054393+0000","last_clean":"2024-04-23T18:18:36.054393+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:18:36.054393+0000","last_undegraded":"2024-04-23T18:18:36.054393+0000","last_fullsized":"2024-04-23T18:18:36.054393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.053423+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:18:36.053423+0000","last_peered":"2024-04-23T18:18:36.053423+0000","last_clean":"2024-04-23T18:18:36.053423+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:18:36.053423+0000","last_undegraded":"2024-04-23T18:18:36.053423+0000","last_fullsized":"2024-04-23T18:18:36.053423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054191+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:18:36.054191+0000","last_peered":"2024-04-23T18:18:36.054191+0000","last_clean":"2024-04-23T18:18:36.054191+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:18:36.054191+0000","last_undegraded":"2024-04-23T18:18:36.054191+0000","last_fullsized":"2024-04-23T18:18:36.054191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997670+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:18:36.997670+0000","last_peered":"2024-04-23T18:18:36.997670+0000","last_clean":"2024-04-23T18:18:36.997670+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:18:36.997670+0000","last_undegraded":"2024-04-23T18:18:36.997670+0000","last_fullsized":"2024-04-23T18:18:36.997670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054190+0000","last_change":"2024-04-23T18:18:35.067581+0000","last_active":"2024-04-23T18:18:36.054190+0000","last_peered":"2024-04-23T18:18:36.054190+0000","last_clean":"2024-04-23T18:18:36.054190+0000","last_became_active":"2024-04-23T18:18:35.066997+0000","last_became_peered":"2024-04-23T18:18:35.066997+0000","last_unstale":"2024-04-23T18:18:36.054190+0000","last_undegraded":"2024-04-23T18:18:36.054190+0000","last_fullsized":"2024-04-23T18:18:36.054190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:06.611678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542792+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:18:36.542792+0000","last_peered":"2024-04-23T18:18:36.542792+0000","last_clean":"2024-04-23T18:18:36.542792+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:18:36.542792+0000","last_undegraded":"2024-04-23T18:18:36.542792+0000","last_fullsized":"2024-04-23T18:18:36.542792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.086394+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:18:39.086394+0000","last_peered":"2024-04-23T18:18:39.086394+0000","last_clean":"2024-04-23T18:18:39.086394+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:18:39.086394+0000","last_undegraded":"2024-04-23T18:18:39.086394+0000","last_fullsized":"2024-04-23T18:18:39.086394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606974+0000","last_change":"2024-04-23T18:18:35.069536+0000","last_active":"2024-04-23T18:18:36.606974+0000","last_peered":"2024-04-23T18:18:36.606974+0000","last_clean":"2024-04-23T18:18:36.606974+0000","last_became_active":"2024-04-23T18:18:35.069169+0000","last_became_peered":"2024-04-23T18:18:35.069169+0000","last_unstale":"2024-04-23T18:18:36.606974+0000","last_undegraded":"2024-04-23T18:18:36.606974+0000","last_fullsized":"2024-04-23T18:18:36.606974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:18:07.101935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:40.090490+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:18:40.090490+0000","last_peered":"2024-04-23T18:18:40.090490+0000","last_clean":"2024-04-23T18:18:40.090490+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:18:40.090490+0000","last_undegraded":"2024-04-23T18:18:40.090490+0000","last_fullsized":"2024-04-23T18:18:40.090490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997494+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:18:36.997494+0000","last_peered":"2024-04-23T18:18:36.997494+0000","last_clean":"2024-04-23T18:18:36.997494+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:18:36.997494+0000","last_undegraded":"2024-04-23T18:18:36.997494+0000","last_fullsized":"2024-04-23T18:18:36.997494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606471+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:18:36.606471+0000","last_peered":"2024-04-23T18:18:36.606471+0000","last_clean":"2024-04-23T18:18:36.606471+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:18:36.606471+0000","last_undegraded":"2024-04-23T18:18:36.606471+0000","last_fullsized":"2024-04-23T18:18:36.606471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.083905+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:18:39.083905+0000","last_peered":"2024-04-23T18:18:39.083905+0000","last_clean":"2024-04-23T18:18:39.083905+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:18:39.083905+0000","last_undegraded":"2024-04-23T18:18:39.083905+0000","last_fullsized":"2024-04-23T18:18:39.083905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606448+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:18:36.606448+0000","last_peered":"2024-04-23T18:18:36.606448+0000","last_clean":"2024-04-23T18:18:36.606448+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:18:36.606448+0000","last_undegraded":"2024-04-23T18:18:36.606448+0000","last_fullsized":"2024-04-23T18:18:36.606448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608042+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:18:36.608042+0000","last_peered":"2024-04-23T18:18:36.608042+0000","last_clean":"2024-04-23T18:18:36.608042+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:18:36.608042+0000","last_undegraded":"2024-04-23T18:18:36.608042+0000","last_fullsized":"2024-04-23T18:18:36.608042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.085952+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:18:39.085952+0000","last_peered":"2024-04-23T18:18:39.085952+0000","last_clean":"2024-04-23T18:18:39.085952+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:18:39.085952+0000","last_undegraded":"2024-04-23T18:18:39.085952+0000","last_fullsized":"2024-04-23T18:18:39.085952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607608+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:18:36.607608+0000","last_peered":"2024-04-23T18:18:36.607608+0000","last_clean":"2024-04-23T18:18:36.607608+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:18:36.607608+0000","last_undegraded":"2024-04-23T18:18:36.607608+0000","last_fullsized":"2024-04-23T18:18:36.607608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074291+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:18:38.074291+0000","last_peered":"2024-04-23T18:18:38.074291+0000","last_clean":"2024-04-23T18:18:38.074291+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:18:38.074291+0000","last_undegraded":"2024-04-23T18:18:38.074291+0000","last_fullsized":"2024-04-23T18:18:38.074291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607090+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:18:36.607090+0000","last_peered":"2024-04-23T18:18:36.607090+0000","last_clean":"2024-04-23T18:18:36.607090+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:18:36.607090+0000","last_undegraded":"2024-04-23T18:18:36.607090+0000","last_fullsized":"2024-04-23T18:18:36.607090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.073860+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:18:38.073860+0000","last_peered":"2024-04-23T18:18:38.073860+0000","last_clean":"2024-04-23T18:18:38.073860+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:18:38.073860+0000","last_undegraded":"2024-04-23T18:18:38.073860+0000","last_fullsized":"2024-04-23T18:18:38.073860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606220+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:18:36.606220+0000","last_peered":"2024-04-23T18:18:36.606220+0000","last_clean":"2024-04-23T18:18:36.606220+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:18:36.606220+0000","last_undegraded":"2024-04-23T18:18:36.606220+0000","last_fullsized":"2024-04-23T18:18:36.606220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.075327+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:18:38.075327+0000","last_peered":"2024-04-23T18:18:38.075327+0000","last_clean":"2024-04-23T18:18:38.075327+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:18:38.075327+0000","last_undegraded":"2024-04-23T18:18:38.075327+0000","last_fullsized":"2024-04-23T18:18:38.075327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054273+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:18:36.054273+0000","last_peered":"2024-04-23T18:18:36.054273+0000","last_clean":"2024-04-23T18:18:36.054273+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:18:36.054273+0000","last_undegraded":"2024-04-23T18:18:36.054273+0000","last_fullsized":"2024-04-23T18:18:36.054273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289004+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:18:36.289004+0000","last_peered":"2024-04-23T18:18:36.289004+0000","last_clean":"2024-04-23T18:18:36.289004+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:18:36.289004+0000","last_undegraded":"2024-04-23T18:18:36.289004+0000","last_fullsized":"2024-04-23T18:18:36.289004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606584+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:18:36.606584+0000","last_peered":"2024-04-23T18:18:36.606584+0000","last_clean":"2024-04-23T18:18:36.606584+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:18:36.606584+0000","last_undegraded":"2024-04-23T18:18:36.606584+0000","last_fullsized":"2024-04-23T18:18:36.606584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074432+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:18:38.074432+0000","last_peered":"2024-04-23T18:18:38.074432+0000","last_clean":"2024-04-23T18:18:38.074432+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:18:38.074432+0000","last_undegraded":"2024-04-23T18:18:38.074432+0000","last_fullsized":"2024-04-23T18:18:38.074432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997807+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:18:36.997807+0000","last_peered":"2024-04-23T18:18:36.997807+0000","last_clean":"2024-04-23T18:18:36.997807+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:18:36.997807+0000","last_undegraded":"2024-04-23T18:18:36.997807+0000","last_fullsized":"2024-04-23T18:18:36.997807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.085154+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:18:39.085154+0000","last_peered":"2024-04-23T18:18:39.085154+0000","last_clean":"2024-04-23T18:18:39.085154+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:18:39.085154+0000","last_undegraded":"2024-04-23T18:18:39.085154+0000","last_fullsized":"2024-04-23T18:18:39.085154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.998029+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:18:36.998029+0000","last_peered":"2024-04-23T18:18:36.998029+0000","last_clean":"2024-04-23T18:18:36.998029+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:18:36.998029+0000","last_undegraded":"2024-04-23T18:18:36.998029+0000","last_fullsized":"2024-04-23T18:18:36.998029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542845+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:18:36.542845+0000","last_peered":"2024-04-23T18:18:36.542845+0000","last_clean":"2024-04-23T18:18:36.542845+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:18:36.542845+0000","last_undegraded":"2024-04-23T18:18:36.542845+0000","last_fullsized":"2024-04-23T18:18:36.542845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997975+0000","last_change":"2024-04-23T18:18:35.051887+0000","last_active":"2024-04-23T18:18:36.997975+0000","last_peered":"2024-04-23T18:18:36.997975+0000","last_clean":"2024-04-23T18:18:36.997975+0000","last_became_active":"2024-04-23T18:18:35.051515+0000","last_became_peered":"2024-04-23T18:18:35.051515+0000","last_unstale":"2024-04-23T18:18:36.997975+0000","last_undegraded":"2024-04-23T18:18:36.997975+0000","last_fullsized":"2024-04-23T18:18:36.997975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:02.144796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607103+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:18:36.607103+0000","last_peered":"2024-04-23T18:18:36.607103+0000","last_clean":"2024-04-23T18:18:36.607103+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:18:36.607103+0000","last_undegraded":"2024-04-23T18:18:36.607103+0000","last_fullsized":"2024-04-23T18:18:36.607103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.998050+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:18:36.998050+0000","last_peered":"2024-04-23T18:18:36.998050+0000","last_clean":"2024-04-23T18:18:36.998050+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:18:36.998050+0000","last_undegraded":"2024-04-23T18:18:36.998050+0000","last_fullsized":"2024-04-23T18:18:36.998050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054261+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:18:36.054261+0000","last_peered":"2024-04-23T18:18:36.054261+0000","last_clean":"2024-04-23T18:18:36.054261+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:18:36.054261+0000","last_undegraded":"2024-04-23T18:18:36.054261+0000","last_fullsized":"2024-04-23T18:18:36.054261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.288980+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:18:36.288980+0000","last_peered":"2024-04-23T18:18:36.288980+0000","last_clean":"2024-04-23T18:18:36.288980+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:18:36.288980+0000","last_undegraded":"2024-04-23T18:18:36.288980+0000","last_fullsized":"2024-04-23T18:18:36.288980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289105+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:18:36.289105+0000","last_peered":"2024-04-23T18:18:36.289105+0000","last_clean":"2024-04-23T18:18:36.289105+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:18:36.289105+0000","last_undegraded":"2024-04-23T18:18:36.289105+0000","last_fullsized":"2024-04-23T18:18:36.289105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607199+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:18:36.607199+0000","last_peered":"2024-04-23T18:18:36.607199+0000","last_clean":"2024-04-23T18:18:36.607199+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:18:36.607199+0000","last_undegraded":"2024-04-23T18:18:36.607199+0000","last_fullsized":"2024-04-23T18:18:36.607199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608055+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:18:36.608055+0000","last_peered":"2024-04-23T18:18:36.608055+0000","last_clean":"2024-04-23T18:18:36.608055+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:18:36.608055+0000","last_undegraded":"2024-04-23T18:18:36.608055+0000","last_fullsized":"2024-04-23T18:18:36.608055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054490+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:18:36.054490+0000","last_peered":"2024-04-23T18:18:36.054490+0000","last_clean":"2024-04-23T18:18:36.054490+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:18:36.054490+0000","last_undegraded":"2024-04-23T18:18:36.054490+0000","last_fullsized":"2024-04-23T18:18:36.054490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289385+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:18:36.289385+0000","last_peered":"2024-04-23T18:18:36.289385+0000","last_clean":"2024-04-23T18:18:36.289385+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:18:36.289385+0000","last_undegraded":"2024-04-23T18:18:36.289385+0000","last_fullsized":"2024-04-23T18:18:36.289385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607406+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:18:36.607406+0000","last_peered":"2024-04-23T18:18:36.607406+0000","last_clean":"2024-04-23T18:18:36.607406+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:18:36.607406+0000","last_undegraded":"2024-04-23T18:18:36.607406+0000","last_fullsized":"2024-04-23T18:18:36.607406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542618+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:18:36.542618+0000","last_peered":"2024-04-23T18:18:36.542618+0000","last_clean":"2024-04-23T18:18:36.542618+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:18:36.542618+0000","last_undegraded":"2024-04-23T18:18:36.542618+0000","last_fullsized":"2024-04-23T18:18:36.542618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607307+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:18:36.607307+0000","last_peered":"2024-04-23T18:18:36.607307+0000","last_clean":"2024-04-23T18:18:36.607307+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:18:36.607307+0000","last_undegraded":"2024-04-23T18:18:36.607307+0000","last_fullsized":"2024-04-23T18:18:36.607307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606290+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:18:36.606290+0000","last_peered":"2024-04-23T18:18:36.606290+0000","last_clean":"2024-04-23T18:18:36.606290+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:18:36.606290+0000","last_undegraded":"2024-04-23T18:18:36.606290+0000","last_fullsized":"2024-04-23T18:18:36.606290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607403+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:18:36.607403+0000","last_peered":"2024-04-23T18:18:36.607403+0000","last_clean":"2024-04-23T18:18:36.607403+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:18:36.607403+0000","last_undegraded":"2024-04-23T18:18:36.607403+0000","last_fullsized":"2024-04-23T18:18:36.607403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054248+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:18:36.054248+0000","last_peered":"2024-04-23T18:18:36.054248+0000","last_clean":"2024-04-23T18:18:36.054248+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:18:36.054248+0000","last_undegraded":"2024-04-23T18:18:36.054248+0000","last_fullsized":"2024-04-23T18:18:36.054248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5501,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":118,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":52932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":455028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1047964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":455846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":455632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1556480,"data_stored":1046600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":455028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":475923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":455118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":457990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182452,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":455312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":455672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1572864,"data_stored":1046872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:18:42.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:18:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 -- 172.21.15.37:0/515352444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64101fe0 msgr2=0x7f3f64102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 --2- 172.21.15.37:0/515352444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64101fe0 0x7f3f64102460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009900 tx=0x7f3f5802a410 comp rx=0 tx=0).stop 2024-04-23T18:18:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 -- 172.21.15.37:0/515352444 shutdown_connections 2024-04-23T18:18:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 --2- 172.21.15.37:0/515352444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f641029a0 0x7f3f641092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 --2- 172.21.15.37:0/515352444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64101fe0 0x7f3f64102460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 --2- 172.21.15.37:0/515352444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f64100e90 0x7f3f641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 -- 172.21.15.37:0/515352444 >> 172.21.15.37:0/515352444 conn(0x7f3f640fc400 msgr2=0x7f3f640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 -- 172.21.15.37:0/515352444 shutdown_connections 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.041+0000 7f3f68be5640 1 -- 172.21.15.37:0/515352444 wait complete. 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 Processor -- start 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 -- start start 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 0x7f3f64198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f64101fe0 0x7f3f641987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f641029a0 0x7f3f6419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6410ca60 con 0x7f3f64100e90 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6410c8e0 con 0x7f3f641029a0 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f68be5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6410cbe0 con 0x7f3f64101fe0 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 0x7f3f64198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f6259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 0x7f3f64198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43204/0 (socket says 172.21.15.37:43204) 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.042+0000 7f3f6259c640 1 -- 172.21.15.37:0/3730205946 learned_addr learned my addr 172.21.15.37:0/3730205946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f62d9d640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f641029a0 0x7f3f6419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f61d9b640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f64101fe0 0x7f3f641987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f6259c640 1 -- 172.21.15.37:0/3730205946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f64101fe0 msgr2=0x7f3f641987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f6259c640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f64101fe0 0x7f3f641987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f6259c640 1 -- 172.21.15.37:0/3730205946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f641029a0 msgr2=0x7f3f6419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f6259c640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f641029a0 0x7f3f6419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f6259c640 1 -- 172.21.15.37:0/3730205946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6419fea0 con 0x7f3f64100e90 2024-04-23T18:18:43.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f6259c640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 0x7f3f64198280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400b4a0 tx=0x7f3f5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:43.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54014070 con 0x7f3f64100e90 2024-04-23T18:18:43.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f54010ac0 con 0x7f3f64100e90 2024-04-23T18:18:43.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f641a0190 con 0x7f3f64100e90 2024-04-23T18:18:43.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54017c30 con 0x7f3f64100e90 2024-04-23T18:18:43.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.043+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f641a0760 con 0x7f3f64100e90 2024-04-23T18:18:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.044+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f641a0a20 con 0x7f3f64100e90 2024-04-23T18:18:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.045+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f54017070 con 0x7f3f64100e90 2024-04-23T18:18:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.046+0000 7f3f537fe640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3f4c0a3c40 0x7f3f4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.046+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3f540cfa60 con 0x7f3f64100e90 2024-04-23T18:18:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.046+0000 7f3f61d9b640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3f4c0a3c40 0x7f3f4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.046+0000 7f3f61d9b640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3f4c0a3c40 0x7f3f4c0a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f58031500 tx=0x7f3f58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:43.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.050+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f5409c5a0 con 0x7f3f64100e90 2024-04-23T18:18:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.239+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f640fe620 con 0x7f3f4c0a3c40 2024-04-23T18:18:43.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.246+0000 7f3f537fe640 1 -- 172.21.15.37:0/3730205946 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258759 (secure 0 0 0) 0x7f3f640fe620 con 0x7f3f4c0a3c40 2024-04-23T18:18:43.242 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3f4c0a3c40 msgr2=0x7f3f4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3f4c0a3c40 0x7f3f4c0a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f58031500 tx=0x7f3f58000bc0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 msgr2=0x7f3f64198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 0x7f3f64198280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400b4a0 tx=0x7f3f5400b960 comp rx=0 tx=0).stop 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 shutdown_connections 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3f4c0a3c40 0x7f3f4c0a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f641029a0 0x7f3f6419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f64101fe0 0x7f3f641987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 --2- 172.21.15.37:0/3730205946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f64100e90 0x7f3f64198280 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 >> 172.21.15.37:0/3730205946 conn(0x7f3f640fc400 msgr2=0x7f3f640fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 shutdown_connections 2024-04-23T18:18:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:43.249+0000 7f3f68be5640 1 -- 172.21.15.37:0/3730205946 wait complete. 2024-04-23T18:18:43.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:43 smithi037 ceph-mon[29404]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-04-23T18:18:43.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:43 smithi037 ceph-mon[29404]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:43.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:43 smithi037 ceph-mon[29404]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:18:43.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:43 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/838118149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:43 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1373165596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.254 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-23T18:18:41.861077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595869,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":146,"num_read_kb":128,"num_write":299,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374724,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567100,"statfs":{"total":1151940427776,"available":1151556710400,"internally_reserved":0,"allocated":13389824,"data_stored":7262985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5501,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":118,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.828725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054481+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:18:36.054481+0000","last_peered":"2024-04-23T18:18:36.054481+0000","last_clean":"2024-04-23T18:18:36.054481+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:18:36.054481+0000","last_undegraded":"2024-04-23T18:18:36.054481+0000","last_fullsized":"2024-04-23T18:18:36.054481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607148+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:18:36.607148+0000","last_peered":"2024-04-23T18:18:36.607148+0000","last_clean":"2024-04-23T18:18:36.607148+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:18:36.607148+0000","last_undegraded":"2024-04-23T18:18:36.607148+0000","last_fullsized":"2024-04-23T18:18:36.607148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607062+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:18:36.607062+0000","last_peered":"2024-04-23T18:18:36.607062+0000","last_clean":"2024-04-23T18:18:36.607062+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:18:36.607062+0000","last_undegraded":"2024-04-23T18:18:36.607062+0000","last_fullsized":"2024-04-23T18:18:36.607062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606681+0000","last_change":"2024-04-23T18:18:35.065476+0000","last_active":"2024-04-23T18:18:36.606681+0000","last_peered":"2024-04-23T18:18:36.606681+0000","last_clean":"2024-04-23T18:18:36.606681+0000","last_became_active":"2024-04-23T18:18:35.065075+0000","last_became_peered":"2024-04-23T18:18:35.065075+0000","last_unstale":"2024-04-23T18:18:36.606681+0000","last_undegraded":"2024-04-23T18:18:36.606681+0000","last_fullsized":"2024-04-23T18:18:36.606681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:37.944188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606176+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:18:36.606176+0000","last_peered":"2024-04-23T18:18:36.606176+0000","last_clean":"2024-04-23T18:18:36.606176+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:18:36.606176+0000","last_undegraded":"2024-04-23T18:18:36.606176+0000","last_fullsized":"2024-04-23T18:18:36.606176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054348+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:18:36.054348+0000","last_peered":"2024-04-23T18:18:36.054348+0000","last_clean":"2024-04-23T18:18:36.054348+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:18:36.054348+0000","last_undegraded":"2024-04-23T18:18:36.054348+0000","last_fullsized":"2024-04-23T18:18:36.054348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997584+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:18:36.997584+0000","last_peered":"2024-04-23T18:18:36.997584+0000","last_clean":"2024-04-23T18:18:36.997584+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:18:36.997584+0000","last_undegraded":"2024-04-23T18:18:36.997584+0000","last_fullsized":"2024-04-23T18:18:36.997584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606246+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:18:36.606246+0000","last_peered":"2024-04-23T18:18:36.606246+0000","last_clean":"2024-04-23T18:18:36.606246+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:18:36.606246+0000","last_undegraded":"2024-04-23T18:18:36.606246+0000","last_fullsized":"2024-04-23T18:18:36.606246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606742+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:18:36.606742+0000","last_peered":"2024-04-23T18:18:36.606742+0000","last_clean":"2024-04-23T18:18:36.606742+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:18:36.606742+0000","last_undegraded":"2024-04-23T18:18:36.606742+0000","last_fullsized":"2024-04-23T18:18:36.606742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054346+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:18:36.054346+0000","last_peered":"2024-04-23T18:18:36.054346+0000","last_clean":"2024-04-23T18:18:36.054346+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:18:36.054346+0000","last_undegraded":"2024-04-23T18:18:36.054346+0000","last_fullsized":"2024-04-23T18:18:36.054346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997899+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:18:36.997899+0000","last_peered":"2024-04-23T18:18:36.997899+0000","last_clean":"2024-04-23T18:18:36.997899+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:18:36.997899+0000","last_undegraded":"2024-04-23T18:18:36.997899+0000","last_fullsized":"2024-04-23T18:18:36.997899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606671+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:18:36.606671+0000","last_peered":"2024-04-23T18:18:36.606671+0000","last_clean":"2024-04-23T18:18:36.606671+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:18:36.606671+0000","last_undegraded":"2024-04-23T18:18:36.606671+0000","last_fullsized":"2024-04-23T18:18:36.606671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607484+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:18:36.607484+0000","last_peered":"2024-04-23T18:18:36.607484+0000","last_clean":"2024-04-23T18:18:36.607484+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:18:36.607484+0000","last_undegraded":"2024-04-23T18:18:36.607484+0000","last_fullsized":"2024-04-23T18:18:36.607484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606827+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:18:36.606827+0000","last_peered":"2024-04-23T18:18:36.606827+0000","last_clean":"2024-04-23T18:18:36.606827+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:18:36.606827+0000","last_undegraded":"2024-04-23T18:18:36.606827+0000","last_fullsized":"2024-04-23T18:18:36.606827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606769+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:18:36.606769+0000","last_peered":"2024-04-23T18:18:36.606769+0000","last_clean":"2024-04-23T18:18:36.606769+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:18:36.606769+0000","last_undegraded":"2024-04-23T18:18:36.606769+0000","last_fullsized":"2024-04-23T18:18:36.606769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606089+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:18:36.606089+0000","last_peered":"2024-04-23T18:18:36.606089+0000","last_clean":"2024-04-23T18:18:36.606089+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:18:36.606089+0000","last_undegraded":"2024-04-23T18:18:36.606089+0000","last_fullsized":"2024-04-23T18:18:36.606089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054598+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:18:36.054598+0000","last_peered":"2024-04-23T18:18:36.054598+0000","last_clean":"2024-04-23T18:18:36.054598+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:18:36.054598+0000","last_undegraded":"2024-04-23T18:18:36.054598+0000","last_fullsized":"2024-04-23T18:18:36.054598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.053508+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:18:36.053508+0000","last_peered":"2024-04-23T18:18:36.053508+0000","last_clean":"2024-04-23T18:18:36.053508+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:18:36.053508+0000","last_undegraded":"2024-04-23T18:18:36.053508+0000","last_fullsized":"2024-04-23T18:18:36.053508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289295+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:18:36.289295+0000","last_peered":"2024-04-23T18:18:36.289295+0000","last_clean":"2024-04-23T18:18:36.289295+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:18:36.289295+0000","last_undegraded":"2024-04-23T18:18:36.289295+0000","last_fullsized":"2024-04-23T18:18:36.289295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.087161+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:18:39.087161+0000","last_peered":"2024-04-23T18:18:39.087161+0000","last_clean":"2024-04-23T18:18:39.087161+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:18:39.087161+0000","last_undegraded":"2024-04-23T18:18:39.087161+0000","last_fullsized":"2024-04-23T18:18:39.087161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607523+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:18:36.607523+0000","last_peered":"2024-04-23T18:18:36.607523+0000","last_clean":"2024-04-23T18:18:36.607523+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:18:36.607523+0000","last_undegraded":"2024-04-23T18:18:36.607523+0000","last_fullsized":"2024-04-23T18:18:36.607523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607793+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:18:36.607793+0000","last_peered":"2024-04-23T18:18:36.607793+0000","last_clean":"2024-04-23T18:18:36.607793+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:18:36.607793+0000","last_undegraded":"2024-04-23T18:18:36.607793+0000","last_fullsized":"2024-04-23T18:18:36.607793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608318+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:18:36.608318+0000","last_peered":"2024-04-23T18:18:36.608318+0000","last_clean":"2024-04-23T18:18:36.608318+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:18:36.608318+0000","last_undegraded":"2024-04-23T18:18:36.608318+0000","last_fullsized":"2024-04-23T18:18:36.608318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606537+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:18:36.606537+0000","last_peered":"2024-04-23T18:18:36.606537+0000","last_clean":"2024-04-23T18:18:36.606537+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:18:36.606537+0000","last_undegraded":"2024-04-23T18:18:36.606537+0000","last_fullsized":"2024-04-23T18:18:36.606537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606578+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:18:36.606578+0000","last_peered":"2024-04-23T18:18:36.606578+0000","last_clean":"2024-04-23T18:18:36.606578+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:18:36.606578+0000","last_undegraded":"2024-04-23T18:18:36.606578+0000","last_fullsized":"2024-04-23T18:18:36.606578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606432+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:18:36.606432+0000","last_peered":"2024-04-23T18:18:36.606432+0000","last_clean":"2024-04-23T18:18:36.606432+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:18:36.606432+0000","last_undegraded":"2024-04-23T18:18:36.606432+0000","last_fullsized":"2024-04-23T18:18:36.606432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.543061+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:18:36.543061+0000","last_peered":"2024-04-23T18:18:36.543061+0000","last_clean":"2024-04-23T18:18:36.543061+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:18:36.543061+0000","last_undegraded":"2024-04-23T18:18:36.543061+0000","last_fullsized":"2024-04-23T18:18:36.543061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.075812+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:18:38.075812+0000","last_peered":"2024-04-23T18:18:38.075812+0000","last_clean":"2024-04-23T18:18:38.075812+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:18:38.075812+0000","last_undegraded":"2024-04-23T18:18:38.075812+0000","last_fullsized":"2024-04-23T18:18:38.075812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606482+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:18:36.606482+0000","last_peered":"2024-04-23T18:18:36.606482+0000","last_clean":"2024-04-23T18:18:36.606482+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:18:36.606482+0000","last_undegraded":"2024-04-23T18:18:36.606482+0000","last_fullsized":"2024-04-23T18:18:36.606482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.083959+0000","last_change":"2024-04-23T18:18:34.049909+0000","last_active":"2024-04-23T18:18:39.083959+0000","last_peered":"2024-04-23T18:18:39.083959+0000","last_clean":"2024-04-23T18:18:39.083959+0000","last_became_active":"2024-04-23T18:18:34.049566+0000","last_became_peered":"2024-04-23T18:18:34.049566+0000","last_unstale":"2024-04-23T18:18:39.083959+0000","last_undegraded":"2024-04-23T18:18:39.083959+0000","last_fullsized":"2024-04-23T18:18:39.083959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:50:06.243216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.543108+0000","last_change":"2024-04-23T18:18:35.065394+0000","last_active":"2024-04-23T18:18:36.543108+0000","last_peered":"2024-04-23T18:18:36.543108+0000","last_clean":"2024-04-23T18:18:36.543108+0000","last_became_active":"2024-04-23T18:18:35.064723+0000","last_became_peered":"2024-04-23T18:18:35.064723+0000","last_unstale":"2024-04-23T18:18:36.543108+0000","last_undegraded":"2024-04-23T18:18:36.543108+0000","last_fullsized":"2024-04-23T18:18:36.543108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:21.430135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.085881+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:18:39.085881+0000","last_peered":"2024-04-23T18:18:39.085881+0000","last_clean":"2024-04-23T18:18:39.085881+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:18:39.085881+0000","last_undegraded":"2024-04-23T18:18:39.085881+0000","last_fullsized":"2024-04-23T18:18:39.085881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606250+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:18:36.606250+0000","last_peered":"2024-04-23T18:18:36.606250+0000","last_clean":"2024-04-23T18:18:36.606250+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:18:36.606250+0000","last_undegraded":"2024-04-23T18:18:36.606250+0000","last_fullsized":"2024-04-23T18:18:36.606250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542578+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:18:36.542578+0000","last_peered":"2024-04-23T18:18:36.542578+0000","last_clean":"2024-04-23T18:18:36.542578+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:18:36.542578+0000","last_undegraded":"2024-04-23T18:18:36.542578+0000","last_fullsized":"2024-04-23T18:18:36.542578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.053407+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:18:36.053407+0000","last_peered":"2024-04-23T18:18:36.053407+0000","last_clean":"2024-04-23T18:18:36.053407+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:18:36.053407+0000","last_undegraded":"2024-04-23T18:18:36.053407+0000","last_fullsized":"2024-04-23T18:18:36.053407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074246+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:18:38.074246+0000","last_peered":"2024-04-23T18:18:38.074246+0000","last_clean":"2024-04-23T18:18:38.074246+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:18:38.074246+0000","last_undegraded":"2024-04-23T18:18:38.074246+0000","last_fullsized":"2024-04-23T18:18:38.074246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606877+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:18:36.606877+0000","last_peered":"2024-04-23T18:18:36.606877+0000","last_clean":"2024-04-23T18:18:36.606877+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:18:36.606877+0000","last_undegraded":"2024-04-23T18:18:36.606877+0000","last_fullsized":"2024-04-23T18:18:36.606877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074905+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:18:38.074905+0000","last_peered":"2024-04-23T18:18:38.074905+0000","last_clean":"2024-04-23T18:18:38.074905+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:18:38.074905+0000","last_undegraded":"2024-04-23T18:18:38.074905+0000","last_fullsized":"2024-04-23T18:18:38.074905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608308+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:18:36.608308+0000","last_peered":"2024-04-23T18:18:36.608308+0000","last_clean":"2024-04-23T18:18:36.608308+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:18:36.608308+0000","last_undegraded":"2024-04-23T18:18:36.608308+0000","last_fullsized":"2024-04-23T18:18:36.608308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.543193+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:18:36.543193+0000","last_peered":"2024-04-23T18:18:36.543193+0000","last_clean":"2024-04-23T18:18:36.543193+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:18:36.543193+0000","last_undegraded":"2024-04-23T18:18:36.543193+0000","last_fullsized":"2024-04-23T18:18:36.543193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606901+0000","last_change":"2024-04-23T18:18:35.067537+0000","last_active":"2024-04-23T18:18:36.606901+0000","last_peered":"2024-04-23T18:18:36.606901+0000","last_clean":"2024-04-23T18:18:36.606901+0000","last_became_active":"2024-04-23T18:18:35.064318+0000","last_became_peered":"2024-04-23T18:18:35.064318+0000","last_unstale":"2024-04-23T18:18:36.606901+0000","last_undegraded":"2024-04-23T18:18:36.606901+0000","last_fullsized":"2024-04-23T18:18:36.606901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:52:35.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.073910+0000","last_change":"2024-04-23T18:18:34.048993+0000","last_active":"2024-04-23T18:18:38.073910+0000","last_peered":"2024-04-23T18:18:38.073910+0000","last_clean":"2024-04-23T18:18:38.073910+0000","last_became_active":"2024-04-23T18:18:34.048558+0000","last_became_peered":"2024-04-23T18:18:34.048558+0000","last_unstale":"2024-04-23T18:18:38.073910+0000","last_undegraded":"2024-04-23T18:18:38.073910+0000","last_fullsized":"2024-04-23T18:18:38.073910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:56:10.483326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606440+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:18:36.606440+0000","last_peered":"2024-04-23T18:18:36.606440+0000","last_clean":"2024-04-23T18:18:36.606440+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:18:36.606440+0000","last_undegraded":"2024-04-23T18:18:36.606440+0000","last_fullsized":"2024-04-23T18:18:36.606440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.086456+0000","last_change":"2024-04-23T18:18:34.047124+0000","last_active":"2024-04-23T18:18:39.086456+0000","last_peered":"2024-04-23T18:18:39.086456+0000","last_clean":"2024-04-23T18:18:39.086456+0000","last_became_active":"2024-04-23T18:18:34.046776+0000","last_became_peered":"2024-04-23T18:18:34.046776+0000","last_unstale":"2024-04-23T18:18:39.086456+0000","last_undegraded":"2024-04-23T18:18:39.086456+0000","last_fullsized":"2024-04-23T18:18:39.086456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:24:33.270309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606840+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:18:36.606840+0000","last_peered":"2024-04-23T18:18:36.606840+0000","last_clean":"2024-04-23T18:18:36.606840+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:18:36.606840+0000","last_undegraded":"2024-04-23T18:18:36.606840+0000","last_fullsized":"2024-04-23T18:18:36.606840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.075576+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:18:38.075576+0000","last_peered":"2024-04-23T18:18:38.075576+0000","last_clean":"2024-04-23T18:18:38.075576+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:18:38.075576+0000","last_undegraded":"2024-04-23T18:18:38.075576+0000","last_fullsized":"2024-04-23T18:18:38.075576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997770+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:18:36.997770+0000","last_peered":"2024-04-23T18:18:36.997770+0000","last_clean":"2024-04-23T18:18:36.997770+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:18:36.997770+0000","last_undegraded":"2024-04-23T18:18:36.997770+0000","last_fullsized":"2024-04-23T18:18:36.997770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:40.094404+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:18:40.094404+0000","last_peered":"2024-04-23T18:18:40.094404+0000","last_clean":"2024-04-23T18:18:40.094404+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:18:40.094404+0000","last_undegraded":"2024-04-23T18:18:40.094404+0000","last_fullsized":"2024-04-23T18:18:40.094404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997490+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:18:36.997490+0000","last_peered":"2024-04-23T18:18:36.997490+0000","last_clean":"2024-04-23T18:18:36.997490+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:18:36.997490+0000","last_undegraded":"2024-04-23T18:18:36.997490+0000","last_fullsized":"2024-04-23T18:18:36.997490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'10","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074438+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:18:38.074438+0000","last_peered":"2024-04-23T18:18:38.074438+0000","last_clean":"2024-04-23T18:18:38.074438+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:18:38.074438+0000","last_undegraded":"2024-04-23T18:18:38.074438+0000","last_fullsized":"2024-04-23T18:18:38.074438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542859+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:18:36.542859+0000","last_peered":"2024-04-23T18:18:36.542859+0000","last_clean":"2024-04-23T18:18:36.542859+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:18:36.542859+0000","last_undegraded":"2024-04-23T18:18:36.542859+0000","last_fullsized":"2024-04-23T18:18:36.542859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606412+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:18:36.606412+0000","last_peered":"2024-04-23T18:18:36.606412+0000","last_clean":"2024-04-23T18:18:36.606412+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:18:36.606412+0000","last_undegraded":"2024-04-23T18:18:36.606412+0000","last_fullsized":"2024-04-23T18:18:36.606412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054393+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:18:36.054393+0000","last_peered":"2024-04-23T18:18:36.054393+0000","last_clean":"2024-04-23T18:18:36.054393+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:18:36.054393+0000","last_undegraded":"2024-04-23T18:18:36.054393+0000","last_fullsized":"2024-04-23T18:18:36.054393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.053423+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:18:36.053423+0000","last_peered":"2024-04-23T18:18:36.053423+0000","last_clean":"2024-04-23T18:18:36.053423+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:18:36.053423+0000","last_undegraded":"2024-04-23T18:18:36.053423+0000","last_fullsized":"2024-04-23T18:18:36.053423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054191+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:18:36.054191+0000","last_peered":"2024-04-23T18:18:36.054191+0000","last_clean":"2024-04-23T18:18:36.054191+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:18:36.054191+0000","last_undegraded":"2024-04-23T18:18:36.054191+0000","last_fullsized":"2024-04-23T18:18:36.054191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997670+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:18:36.997670+0000","last_peered":"2024-04-23T18:18:36.997670+0000","last_clean":"2024-04-23T18:18:36.997670+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:18:36.997670+0000","last_undegraded":"2024-04-23T18:18:36.997670+0000","last_fullsized":"2024-04-23T18:18:36.997670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054190+0000","last_change":"2024-04-23T18:18:35.067581+0000","last_active":"2024-04-23T18:18:36.054190+0000","last_peered":"2024-04-23T18:18:36.054190+0000","last_clean":"2024-04-23T18:18:36.054190+0000","last_became_active":"2024-04-23T18:18:35.066997+0000","last_became_peered":"2024-04-23T18:18:35.066997+0000","last_unstale":"2024-04-23T18:18:36.054190+0000","last_undegraded":"2024-04-23T18:18:36.054190+0000","last_fullsized":"2024-04-23T18:18:36.054190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:06.611678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542792+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:18:36.542792+0000","last_peered":"2024-04-23T18:18:36.542792+0000","last_clean":"2024-04-23T18:18:36.542792+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:18:36.542792+0000","last_undegraded":"2024-04-23T18:18:36.542792+0000","last_fullsized":"2024-04-23T18:18:36.542792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.086394+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:18:39.086394+0000","last_peered":"2024-04-23T18:18:39.086394+0000","last_clean":"2024-04-23T18:18:39.086394+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:18:39.086394+0000","last_undegraded":"2024-04-23T18:18:39.086394+0000","last_fullsized":"2024-04-23T18:18:39.086394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606974+0000","last_change":"2024-04-23T18:18:35.069536+0000","last_active":"2024-04-23T18:18:36.606974+0000","last_peered":"2024-04-23T18:18:36.606974+0000","last_clean":"2024-04-23T18:18:36.606974+0000","last_became_active":"2024-04-23T18:18:35.069169+0000","last_became_peered":"2024-04-23T18:18:35.069169+0000","last_unstale":"2024-04-23T18:18:36.606974+0000","last_undegraded":"2024-04-23T18:18:36.606974+0000","last_fullsized":"2024-04-23T18:18:36.606974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:18:07.101935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:40.090490+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:18:40.090490+0000","last_peered":"2024-04-23T18:18:40.090490+0000","last_clean":"2024-04-23T18:18:40.090490+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:18:40.090490+0000","last_undegraded":"2024-04-23T18:18:40.090490+0000","last_fullsized":"2024-04-23T18:18:40.090490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997494+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:18:36.997494+0000","last_peered":"2024-04-23T18:18:36.997494+0000","last_clean":"2024-04-23T18:18:36.997494+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:18:36.997494+0000","last_undegraded":"2024-04-23T18:18:36.997494+0000","last_fullsized":"2024-04-23T18:18:36.997494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606471+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:18:36.606471+0000","last_peered":"2024-04-23T18:18:36.606471+0000","last_clean":"2024-04-23T18:18:36.606471+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:18:36.606471+0000","last_undegraded":"2024-04-23T18:18:36.606471+0000","last_fullsized":"2024-04-23T18:18:36.606471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.083905+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:18:39.083905+0000","last_peered":"2024-04-23T18:18:39.083905+0000","last_clean":"2024-04-23T18:18:39.083905+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:18:39.083905+0000","last_undegraded":"2024-04-23T18:18:39.083905+0000","last_fullsized":"2024-04-23T18:18:39.083905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606448+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:18:36.606448+0000","last_peered":"2024-04-23T18:18:36.606448+0000","last_clean":"2024-04-23T18:18:36.606448+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:18:36.606448+0000","last_undegraded":"2024-04-23T18:18:36.606448+0000","last_fullsized":"2024-04-23T18:18:36.606448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608042+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:18:36.608042+0000","last_peered":"2024-04-23T18:18:36.608042+0000","last_clean":"2024-04-23T18:18:36.608042+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:18:36.608042+0000","last_undegraded":"2024-04-23T18:18:36.608042+0000","last_fullsized":"2024-04-23T18:18:36.608042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.085952+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:18:39.085952+0000","last_peered":"2024-04-23T18:18:39.085952+0000","last_clean":"2024-04-23T18:18:39.085952+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:18:39.085952+0000","last_undegraded":"2024-04-23T18:18:39.085952+0000","last_fullsized":"2024-04-23T18:18:39.085952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607608+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:18:36.607608+0000","last_peered":"2024-04-23T18:18:36.607608+0000","last_clean":"2024-04-23T18:18:36.607608+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:18:36.607608+0000","last_undegraded":"2024-04-23T18:18:36.607608+0000","last_fullsized":"2024-04-23T18:18:36.607608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074291+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:18:38.074291+0000","last_peered":"2024-04-23T18:18:38.074291+0000","last_clean":"2024-04-23T18:18:38.074291+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:18:38.074291+0000","last_undegraded":"2024-04-23T18:18:38.074291+0000","last_fullsized":"2024-04-23T18:18:38.074291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607090+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:18:36.607090+0000","last_peered":"2024-04-23T18:18:36.607090+0000","last_clean":"2024-04-23T18:18:36.607090+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:18:36.607090+0000","last_undegraded":"2024-04-23T18:18:36.607090+0000","last_fullsized":"2024-04-23T18:18:36.607090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.073860+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:18:38.073860+0000","last_peered":"2024-04-23T18:18:38.073860+0000","last_clean":"2024-04-23T18:18:38.073860+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:18:38.073860+0000","last_undegraded":"2024-04-23T18:18:38.073860+0000","last_fullsized":"2024-04-23T18:18:38.073860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606220+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:18:36.606220+0000","last_peered":"2024-04-23T18:18:36.606220+0000","last_clean":"2024-04-23T18:18:36.606220+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:18:36.606220+0000","last_undegraded":"2024-04-23T18:18:36.606220+0000","last_fullsized":"2024-04-23T18:18:36.606220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.075327+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:18:38.075327+0000","last_peered":"2024-04-23T18:18:38.075327+0000","last_clean":"2024-04-23T18:18:38.075327+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:18:38.075327+0000","last_undegraded":"2024-04-23T18:18:38.075327+0000","last_fullsized":"2024-04-23T18:18:38.075327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054273+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:18:36.054273+0000","last_peered":"2024-04-23T18:18:36.054273+0000","last_clean":"2024-04-23T18:18:36.054273+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:18:36.054273+0000","last_undegraded":"2024-04-23T18:18:36.054273+0000","last_fullsized":"2024-04-23T18:18:36.054273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289004+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:18:36.289004+0000","last_peered":"2024-04-23T18:18:36.289004+0000","last_clean":"2024-04-23T18:18:36.289004+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:18:36.289004+0000","last_undegraded":"2024-04-23T18:18:36.289004+0000","last_fullsized":"2024-04-23T18:18:36.289004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606584+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:18:36.606584+0000","last_peered":"2024-04-23T18:18:36.606584+0000","last_clean":"2024-04-23T18:18:36.606584+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:18:36.606584+0000","last_undegraded":"2024-04-23T18:18:36.606584+0000","last_fullsized":"2024-04-23T18:18:36.606584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:38.074432+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:18:38.074432+0000","last_peered":"2024-04-23T18:18:38.074432+0000","last_clean":"2024-04-23T18:18:38.074432+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:18:38.074432+0000","last_undegraded":"2024-04-23T18:18:38.074432+0000","last_fullsized":"2024-04-23T18:18:38.074432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997807+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:18:36.997807+0000","last_peered":"2024-04-23T18:18:36.997807+0000","last_clean":"2024-04-23T18:18:36.997807+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:18:36.997807+0000","last_undegraded":"2024-04-23T18:18:36.997807+0000","last_fullsized":"2024-04-23T18:18:36.997807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:39.085154+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:18:39.085154+0000","last_peered":"2024-04-23T18:18:39.085154+0000","last_clean":"2024-04-23T18:18:39.085154+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:18:39.085154+0000","last_undegraded":"2024-04-23T18:18:39.085154+0000","last_fullsized":"2024-04-23T18:18:39.085154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.998029+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:18:36.998029+0000","last_peered":"2024-04-23T18:18:36.998029+0000","last_clean":"2024-04-23T18:18:36.998029+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:18:36.998029+0000","last_undegraded":"2024-04-23T18:18:36.998029+0000","last_fullsized":"2024-04-23T18:18:36.998029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542845+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:18:36.542845+0000","last_peered":"2024-04-23T18:18:36.542845+0000","last_clean":"2024-04-23T18:18:36.542845+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:18:36.542845+0000","last_undegraded":"2024-04-23T18:18:36.542845+0000","last_fullsized":"2024-04-23T18:18:36.542845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.997975+0000","last_change":"2024-04-23T18:18:35.051887+0000","last_active":"2024-04-23T18:18:36.997975+0000","last_peered":"2024-04-23T18:18:36.997975+0000","last_clean":"2024-04-23T18:18:36.997975+0000","last_became_active":"2024-04-23T18:18:35.051515+0000","last_became_peered":"2024-04-23T18:18:35.051515+0000","last_unstale":"2024-04-23T18:18:36.997975+0000","last_undegraded":"2024-04-23T18:18:36.997975+0000","last_fullsized":"2024-04-23T18:18:36.997975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:02.144796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607103+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:18:36.607103+0000","last_peered":"2024-04-23T18:18:36.607103+0000","last_clean":"2024-04-23T18:18:36.607103+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:18:36.607103+0000","last_undegraded":"2024-04-23T18:18:36.607103+0000","last_fullsized":"2024-04-23T18:18:36.607103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.998050+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:18:36.998050+0000","last_peered":"2024-04-23T18:18:36.998050+0000","last_clean":"2024-04-23T18:18:36.998050+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:18:36.998050+0000","last_undegraded":"2024-04-23T18:18:36.998050+0000","last_fullsized":"2024-04-23T18:18:36.998050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054261+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:18:36.054261+0000","last_peered":"2024-04-23T18:18:36.054261+0000","last_clean":"2024-04-23T18:18:36.054261+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:18:36.054261+0000","last_undegraded":"2024-04-23T18:18:36.054261+0000","last_fullsized":"2024-04-23T18:18:36.054261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.288980+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:18:36.288980+0000","last_peered":"2024-04-23T18:18:36.288980+0000","last_clean":"2024-04-23T18:18:36.288980+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:18:36.288980+0000","last_undegraded":"2024-04-23T18:18:36.288980+0000","last_fullsized":"2024-04-23T18:18:36.288980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289105+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:18:36.289105+0000","last_peered":"2024-04-23T18:18:36.289105+0000","last_clean":"2024-04-23T18:18:36.289105+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:18:36.289105+0000","last_undegraded":"2024-04-23T18:18:36.289105+0000","last_fullsized":"2024-04-23T18:18:36.289105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607199+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:18:36.607199+0000","last_peered":"2024-04-23T18:18:36.607199+0000","last_clean":"2024-04-23T18:18:36.607199+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:18:36.607199+0000","last_undegraded":"2024-04-23T18:18:36.607199+0000","last_fullsized":"2024-04-23T18:18:36.607199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.608055+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:18:36.608055+0000","last_peered":"2024-04-23T18:18:36.608055+0000","last_clean":"2024-04-23T18:18:36.608055+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:18:36.608055+0000","last_undegraded":"2024-04-23T18:18:36.608055+0000","last_fullsized":"2024-04-23T18:18:36.608055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054490+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:18:36.054490+0000","last_peered":"2024-04-23T18:18:36.054490+0000","last_clean":"2024-04-23T18:18:36.054490+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:18:36.054490+0000","last_undegraded":"2024-04-23T18:18:36.054490+0000","last_fullsized":"2024-04-23T18:18:36.054490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.289385+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:18:36.289385+0000","last_peered":"2024-04-23T18:18:36.289385+0000","last_clean":"2024-04-23T18:18:36.289385+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:18:36.289385+0000","last_undegraded":"2024-04-23T18:18:36.289385+0000","last_fullsized":"2024-04-23T18:18:36.289385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607406+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:18:36.607406+0000","last_peered":"2024-04-23T18:18:36.607406+0000","last_clean":"2024-04-23T18:18:36.607406+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:18:36.607406+0000","last_undegraded":"2024-04-23T18:18:36.607406+0000","last_fullsized":"2024-04-23T18:18:36.607406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.542618+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:18:36.542618+0000","last_peered":"2024-04-23T18:18:36.542618+0000","last_clean":"2024-04-23T18:18:36.542618+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:18:36.542618+0000","last_undegraded":"2024-04-23T18:18:36.542618+0000","last_fullsized":"2024-04-23T18:18:36.542618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607307+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:18:36.607307+0000","last_peered":"2024-04-23T18:18:36.607307+0000","last_clean":"2024-04-23T18:18:36.607307+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:18:36.607307+0000","last_undegraded":"2024-04-23T18:18:36.607307+0000","last_fullsized":"2024-04-23T18:18:36.607307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.606290+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:18:36.606290+0000","last_peered":"2024-04-23T18:18:36.606290+0000","last_clean":"2024-04-23T18:18:36.606290+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:18:36.606290+0000","last_undegraded":"2024-04-23T18:18:36.606290+0000","last_fullsized":"2024-04-23T18:18:36.606290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.607403+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:18:36.607403+0000","last_peered":"2024-04-23T18:18:36.607403+0000","last_clean":"2024-04-23T18:18:36.607403+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:18:36.607403+0000","last_undegraded":"2024-04-23T18:18:36.607403+0000","last_fullsized":"2024-04-23T18:18:36.607403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:18:36.054248+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:18:36.054248+0000","last_peered":"2024-04-23T18:18:36.054248+0000","last_clean":"2024-04-23T18:18:36.054248+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:18:36.054248+0000","last_undegraded":"2024-04-23T18:18:36.054248+0000","last_fullsized":"2024-04-23T18:18:36.054248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5501,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":118,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":52932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":455028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1047964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":455846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":455632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1556480,"data_stored":1046600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":455028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":475923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":455118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":457990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182452,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":455312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":455672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1572864,"data_stored":1046872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:18:43.257 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T18:18:43.258 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T18:18:43.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T18:18:43.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:43 smithi172 ceph-mon[32456]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-04-23T18:18:43.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:43 smithi172 ceph-mon[32456]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:43.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:43 smithi172 ceph-mon[32456]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:18:43.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:43 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/838118149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:43 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1373165596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:43 smithi173 ceph-mon[32906]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-04-23T18:18:43.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:43 smithi173 ceph-mon[32906]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:43.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:43 smithi173 ceph-mon[32906]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:18:43.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:43 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/838118149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:43 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1373165596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:44.334 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:44 smithi037 ceph-mon[29404]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:44.335 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:44 smithi037 ceph-mon[29404]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:44 smithi172 ceph-mon[32456]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:44 smithi172 ceph-mon[32456]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:44.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:44 smithi173 ceph-mon[32906]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:44.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:44 smithi173 ceph-mon[32906]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 -- 172.21.15.37:0/2354494924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 msgr2=0x7fe91c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2354494924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c109e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe91400b0d0 tx=0x7fe91400b3e0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.183 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:45 smithi037 ceph-mon[29404]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2024-04-23T18:18:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 -- 172.21.15.37:0/2354494924 shutdown_connections 2024-04-23T18:18:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2354494924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c109e10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2354494924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 0x7fe91c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2354494924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe91c102bc0 0x7fe91c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 -- 172.21.15.37:0/2354494924 >> 172.21.15.37:0/2354494924 conn(0x7fe91c0fcf60 msgr2=0x7fe91c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 -- 172.21.15.37:0/2354494924 shutdown_connections 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.189+0000 7fe9222e8700 1 -- 172.21.15.37:0/2354494924 wait complete. 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 Processor -- start 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 -- start start 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 0x7fe91c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe91c102bc0 0x7fe91c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c10d440 con 0x7fe91c103560 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c10d2c0 con 0x7fe91c102bc0 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.190+0000 7fe9222e8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c10d5c0 con 0x7fe91c1019c0 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c19ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33090/0 (socket says 172.21.15.37:33090) 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 -- 172.21.15.37:0/2934220773 learned_addr learned my addr 172.21.15.37:0/2934220773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe91bfff700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 0x7fe91c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe91b7fe700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe91c102bc0 0x7fe91c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 -- 172.21.15.37:0/2934220773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 msgr2=0x7fe91c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 0x7fe91c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 -- 172.21.15.37:0/2934220773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe91c102bc0 msgr2=0x7fe91c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe91c102bc0 0x7fe91c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 -- 172.21.15.37:0/2934220773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c103de0 con 0x7fe91c103560 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe91bfff700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 0x7fe91c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe920885700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c19ffd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe914009fd0 tx=0x7fe914004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91401f030 con 0x7fe91c103560 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.191+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe914003a00 con 0x7fe91c103560 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.192+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914004780 con 0x7fe91c103560 2024-04-23T18:18:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.192+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c104070 con 0x7fe91c103560 2024-04-23T18:18:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.192+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c1045a0 con 0x7fe91c103560 2024-04-23T18:18:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.193+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c050f80 con 0x7fe91c103560 2024-04-23T18:18:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.194+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe914003bb0 con 0x7fe91c103560 2024-04-23T18:18:45.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.195+0000 7fe9197fa700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe9040a3e00 0x7fe9040a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:45.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.195+0000 7fe91bfff700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe9040a3e00 0x7fe9040a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:45.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.195+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe91400f020 con 0x7fe91c103560 2024-04-23T18:18:45.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.195+0000 7fe91bfff700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe9040a3e00 0x7fe9040a62c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe91c102a20 tx=0x7fe90c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:45.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.198+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe914029020 con 0x7fe91c103560 2024-04-23T18:18:45.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:45 smithi173 ceph-mon[32906]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2024-04-23T18:18:45.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.492+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe91c104a50 con 0x7fe91c103560 2024-04-23T18:18:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.494+0000 7fe9197fa700 1 -- 172.21.15.37:0/2934220773 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7fe914011e50 con 0x7fe91c103560 2024-04-23T18:18:45.488 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-23T18:18:45.488 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCF+ydmTJFsHRAAQdyCXzY15GcumRiNbhszKQ== 2024-04-23T18:18:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe9040a3e00 msgr2=0x7fe9040a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe9040a3e00 0x7fe9040a62c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe91c102a20 tx=0x7fe90c00b410 comp rx=0 tx=0).stop 2024-04-23T18:18:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 msgr2=0x7fe91c19ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c19ffd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe914009fd0 tx=0x7fe914004d50 comp rx=0 tx=0).stop 2024-04-23T18:18:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 shutdown_connections 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe9040a3e00 0x7fe9040a62c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91c103560 0x7fe91c19ffd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe91c1019c0 0x7fe91c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 --2- 172.21.15.37:0/2934220773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe91c102bc0 0x7fe91c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 >> 172.21.15.37:0/2934220773 conn(0x7fe91c0fcf60 msgr2=0x7fe91c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 shutdown_connections 2024-04-23T18:18:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:45.497+0000 7fe9222e8700 1 -- 172.21.15.37:0/2934220773 wait complete. 2024-04-23T18:18:45.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:45 smithi172 ceph-mon[32456]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2024-04-23T18:18:46.098 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:18:46.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T18:18:46.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T18:18:46.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:46 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2934220773' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:18:46.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:46 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2934220773' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:18:46.135 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T18:18:46.136 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T18:18:46.136 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T18:18:46.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:46 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2934220773' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:18:46.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:46 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2934220773' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:18:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:46.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:46 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2934220773' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:18:46.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:46 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2934220773' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:18:47.827 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:47 smithi037 ceph-mon[29404]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.8 KiB/s wr, 19 op/s 2024-04-23T18:18:47.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:47 smithi172 ceph-mon[32456]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.8 KiB/s wr, 19 op/s 2024-04-23T18:18:47.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:47 smithi173 ceph-mon[32906]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.8 KiB/s wr, 19 op/s 2024-04-23T18:18:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/59686122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06b0f0 msgr2=0x7f7fec10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/59686122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06b0f0 0x7f7fec10f900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400b0a0 tx=0x7f7fe400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/59686122 shutdown_connections 2024-04-23T18:18:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/59686122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06b0f0 0x7f7fec10f900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/59686122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fec108a30 0x7f7fec108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/59686122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fec06a750 0x7f7fec06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/59686122 >> 172.21.15.37:0/59686122 conn(0x7f7fec0788f0 msgr2=0x7f7fec078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/59686122 shutdown_connections 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.065+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/59686122 wait complete. 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 Processor -- start 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 -- start start 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 0x7f7fec1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fec06b0f0 0x7f7fec199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fec108a30 0x7f7fec19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec112c70 con 0x7f7fec06a750 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec112af0 con 0x7f7fec108a30 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff2d3f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec112df0 con 0x7f7fec06b0f0 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 0x7f7fec1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 0x7f7fec1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33112/0 (socket says 172.21.15.37:33112) 2024-04-23T18:18:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 -- 172.21.15.37:0/4151454012 learned_addr learned my addr 172.21.15.37:0/4151454012 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7febfff700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fec06b0f0 0x7f7fec199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 -- 172.21.15.37:0/4151454012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fec06b0f0 msgr2=0x7f7fec199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fec06b0f0 0x7f7fec199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 -- 172.21.15.37:0/4151454012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fec108a30 msgr2=0x7f7fec19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fec108a30 0x7f7fec19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.066+0000 7f7ff0adb700 1 -- 172.21.15.37:0/4151454012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec19e1d0 con 0x7f7fec06a750 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.067+0000 7f7ff0adb700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 0x7f7fec1991c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c920 tx=0x7f7fe000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.067+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0007b00 con 0x7f7fec06a750 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.067+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe0004ce0 con 0x7f7fec06a750 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.067+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec19e4c0 con 0x7f7fec06a750 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.067+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0005650 con 0x7f7fec06a750 2024-04-23T18:18:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.067+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a5d40 con 0x7f7fec06a750 2024-04-23T18:18:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.068+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec069730 con 0x7f7fec06a750 2024-04-23T18:18:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.069+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7fe0004750 con 0x7f7fec06a750 2024-04-23T18:18:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.070+0000 7f7fe9ffb700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7fdc0a3cd0 0x7f7fdc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.070+0000 7f7febfff700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7fdc0a3cd0 0x7f7fdc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.070+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7fe00d0340 con 0x7f7fec06a750 2024-04-23T18:18:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.070+0000 7f7febfff700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7fdc0a3cd0 0x7f7fdc0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006fd0 tx=0x7f7fd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:48.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.074+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fe0014030 con 0x7f7fec06a750 2024-04-23T18:18:48.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.343+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7fec02df80 con 0x7f7fec06a750 2024-04-23T18:18:48.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.348+0000 7f7fe9ffb700 1 -- 172.21.15.37:0/4151454012 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7f7fe009cd70 con 0x7f7fec06a750 2024-04-23T18:18:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7fdc0a3cd0 msgr2=0x7f7fdc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7fdc0a3cd0 0x7f7fdc0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006fd0 tx=0x7f7fd8008040 comp rx=0 tx=0).stop 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 msgr2=0x7f7fec1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 0x7f7fec1991c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c920 tx=0x7f7fe000cce0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 shutdown_connections 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7fdc0a3cd0 0x7f7fdc0a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fec06a750 0x7f7fec1991c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fec108a30 0x7f7fec19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 --2- 172.21.15.37:0/4151454012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fec06b0f0 0x7f7fec199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 >> 172.21.15.37:0/4151454012 conn(0x7f7fec0788f0 msgr2=0x7f7fec0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.350+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 shutdown_connections 2024-04-23T18:18:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:48.351+0000 7f7ff2d3f700 1 -- 172.21.15.37:0/4151454012 wait complete. 2024-04-23T18:18:48.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:48 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4151454012' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:18:48.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:48 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4151454012' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:18:48.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:48 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4151454012' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:18:49.065 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":3159991179},{"type":"v1","addr":"172.21.15.37:6801","nonce":3159991179}]},"active_addr":"172.21.15.37:6801/3159991179","active_change":"2024-04-23T18:11:39.779586+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2883572010}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2775309719}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2228885295}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":906783050}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":105420574}]}]} 2024-04-23T18:18:49.072 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T18:18:49.073 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T18:18:49.073 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:18:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:49.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:49 smithi037 ceph-mon[29404]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-23T18:18:49.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:49 smithi172 ceph-mon[32456]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-23T18:18:49.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:49 smithi173 ceph-mon[32906]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-23T18:18:50.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 -- 172.21.15.37:0/3701466924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94102a90 msgr2=0x7f3a94102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3701466924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94102a90 0x7f3a94102f10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009990 tx=0x7f3a84009ca0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 -- 172.21.15.37:0/3701466924 shutdown_connections 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3701466924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a94103450 0x7f3a9410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3701466924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a94108a50 0x7f3a94108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3701466924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94102a90 0x7f3a94102f10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 -- 172.21.15.37:0/3701466924 >> 172.21.15.37:0/3701466924 conn(0x7f3a94079ba0 msgr2=0x7f3a94079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 -- 172.21.15.37:0/3701466924 shutdown_connections 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 -- 172.21.15.37:0/3701466924 wait complete. 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 Processor -- start 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.760+0000 7f3a9a847700 1 -- start start 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a9a847700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a94102a90 0x7f3a94074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a9a847700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 0x7f3a94075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a9a847700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a94108a50 0x7f3a940757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a9a847700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112cd0 con 0x7f3a94103450 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a9a847700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112b50 con 0x7f3a94102a90 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a9a847700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112e50 con 0x7f3a94108a50 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 0x7f3a94075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 0x7f3a94075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33132/0 (socket says 172.21.15.37:33132) 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 -- 172.21.15.37:0/3084779952 learned_addr learned my addr 172.21.15.37:0/3084779952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a93fff700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a94102a90 0x7f3a94074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a98de4700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a94108a50 0x7f3a940757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 -- 172.21.15.37:0/3084779952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a94108a50 msgr2=0x7f3a940757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a94108a50 0x7f3a940757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 -- 172.21.15.37:0/3084779952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a94102a90 msgr2=0x7f3a94074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a94102a90 0x7f3a94074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 -- 172.21.15.37:0/3084779952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a941a7710 con 0x7f3a94103450 2024-04-23T18:18:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.761+0000 7f3a937fe700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 0x7f3a94075280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005e80 tx=0x7f3a84005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.762+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84027070 con 0x7f3a94103450 2024-04-23T18:18:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.762+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a84004680 con 0x7f3a94103450 2024-04-23T18:18:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.762+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84022b90 con 0x7f3a94103450 2024-04-23T18:18:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.762+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a941a79a0 con 0x7f3a94103450 2024-04-23T18:18:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.762+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a941a7e30 con 0x7f3a94103450 2024-04-23T18:18:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.763+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a94069730 con 0x7f3a94103450 2024-04-23T18:18:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.764+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a8402a070 con 0x7f3a94103450 2024-04-23T18:18:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.765+0000 7f3a917fa700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a800a3cd0 0x7f3a800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.765+0000 7f3a93fff700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a800a3cd0 0x7f3a800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.765+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3a840d1b20 con 0x7f3a94103450 2024-04-23T18:18:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.765+0000 7f3a93fff700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a800a3cd0 0x7f3a800a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005950 tx=0x7f3a7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.769+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a84031070 con 0x7f3a94103450 2024-04-23T18:18:50.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.980+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a9402e140 con 0x7f3a94103450 2024-04-23T18:18:50.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.982+0000 7f3a917fa700 1 -- 172.21.15.37:0/3084779952 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22243 (secure 0 0 0) 0x7f3a84013070 con 0x7f3a94103450 2024-04-23T18:18:50.975 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:50.975 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:18:36.045489+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:18:50.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a800a3cd0 msgr2=0x7f3a800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:50.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a800a3cd0 0x7f3a800a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005950 tx=0x7f3a7c00b410 comp rx=0 tx=0).stop 2024-04-23T18:18:50.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 msgr2=0x7f3a94075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:50.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 0x7f3a94075280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005e80 tx=0x7f3a84005eb0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 shutdown_connections 2024-04-23T18:18:50.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a800a3cd0 0x7f3a800a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a94108a50 0x7f3a940757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a94102a90 0x7f3a94074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 --2- 172.21.15.37:0/3084779952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a94103450 0x7f3a94075280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:50.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 >> 172.21.15.37:0/3084779952 conn(0x7f3a94079ba0 msgr2=0x7f3a94101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:50.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 shutdown_connections 2024-04-23T18:18:50.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:50.984+0000 7f3a9a847700 1 -- 172.21.15.37:0/3084779952 wait complete. 2024-04-23T18:18:51.634 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:51 smithi037 ceph-mon[29404]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-23T18:18:51.634 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:51 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3084779952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:51.635 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T18:18:51.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:18:51.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:51 smithi172 ceph-mon[32456]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-23T18:18:51.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:51 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3084779952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:51.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:51 smithi173 ceph-mon[32906]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-23T18:18:51.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:51 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3084779952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:51.882 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 -- 172.21.15.37:0/3609112513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 msgr2=0x7f945010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 --2- 172.21.15.37:0/3609112513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010e830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f944800b0a0 tx=0x7f944800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 -- 172.21.15.37:0/3609112513 shutdown_connections 2024-04-23T18:18:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 --2- 172.21.15.37:0/3609112513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010e830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 --2- 172.21.15.37:0/3609112513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9450107960 0x7f9450107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 --2- 172.21.15.37:0/3609112513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 0x7f9450101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 -- 172.21.15.37:0/3609112513 >> 172.21.15.37:0/3609112513 conn(0x7f94500789e0 msgr2=0x7f9450078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 -- 172.21.15.37:0/3609112513 shutdown_connections 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 -- 172.21.15.37:0/3609112513 wait complete. 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 Processor -- start 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.456+0000 7f9457f94700 1 -- start start 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9457f94700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 0x7f945010f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9457f94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9457f94700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9450107960 0x7f945010fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9457f94700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9450112c80 con 0x7f9450102320 2024-04-23T18:18:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9457f94700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9450112b00 con 0x7f9450107960 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9457f94700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9450112e00 con 0x7f9450101960 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9455d30700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 0x7f945010f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33142/0 (socket says 172.21.15.37:33142) 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 -- 172.21.15.37:0/383644677 learned_addr learned my addr 172.21.15.37:0/383644677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9456531700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9450107960 0x7f945010fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 -- 172.21.15.37:0/383644677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 msgr2=0x7f945010f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 0x7f945010f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 -- 172.21.15.37:0/383644677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9450107960 msgr2=0x7f945010fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9450107960 0x7f945010fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 -- 172.21.15.37:0/383644677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94501a7680 con 0x7f9450102320 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9455d30700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 0x7f945010f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f945552f700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010f680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f944000d950 tx=0x7f944000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.457+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440016070 con 0x7f9450102320 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.458+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944000fd40 con 0x7f9450102320 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.458+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94501a7970 con 0x7f9450102320 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.458+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944000e3b0 con 0x7f9450102320 2024-04-23T18:18:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.458+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501a7f00 con 0x7f9450102320 2024-04-23T18:18:53.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.459+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9450069730 con 0x7f9450102320 2024-04-23T18:18:53.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.460+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9440019070 con 0x7f9450102320 2024-04-23T18:18:53.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.460+0000 7f9446ffd700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f943c0a3d50 0x7f943c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:53.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.460+0000 7f9455d30700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f943c0a3d50 0x7f943c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:53.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.460+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f94400cfdd0 con 0x7f9450102320 2024-04-23T18:18:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.461+0000 7f9455d30700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f943c0a3d50 0x7f943c0a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f944c009de0 tx=0x7f944c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:53.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.464+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f944009c800 con 0x7f9450102320 2024-04-23T18:18:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.673+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9450109f60 con 0x7f9450102320 2024-04-23T18:18:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.674+0000 7f9446ffd700 1 -- 172.21.15.37:0/383644677 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22243 (secure 0 0 0) 0x7f944009c100 con 0x7f9450102320 2024-04-23T18:18:53.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:18:53.668 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:18:36.045489+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.676+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f943c0a3d50 msgr2=0x7f943c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.676+0000 7f9457f94700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f943c0a3d50 0x7f943c0a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f944c009de0 tx=0x7f944c009190 comp rx=0 tx=0).stop 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.676+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 msgr2=0x7f945010f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.676+0000 7f9457f94700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010f680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f944000d950 tx=0x7f944000dc60 comp rx=0 tx=0).stop 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 shutdown_connections 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f943c0a3d50 0x7f943c0a6210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9450107960 0x7f945010fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9450101960 0x7f945010f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 --2- 172.21.15.37:0/383644677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9450102320 0x7f945010f680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 >> 172.21.15.37:0/383644677 conn(0x7f94500789e0 msgr2=0x7f94501044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 shutdown_connections 2024-04-23T18:18:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:18:53.677+0000 7f9457f94700 1 -- 172.21.15.37:0/383644677 wait complete. 2024-04-23T18:18:53.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:53 smithi172 ceph-mon[32456]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-23T18:18:53.847 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:53 smithi173 ceph-mon[32906]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-23T18:18:53.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:53 smithi037 ceph-mon[29404]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-23T18:18:54.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T18:18:54.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T18:18:54.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T18:18:54.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T18:18:54.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T18:18:54.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T18:18:54.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-23T18:18:54.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-23T18:18:54.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-23T18:18:54.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-23T18:18:54.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-23T18:18:54.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-23T18:18:54.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:54 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/383644677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:54.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:54 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/383644677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:54.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:54 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/383644677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:18:55.020 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:55.020 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:55.032 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:55.050 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:55.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:55 smithi037 ceph-mon[29404]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-23T18:18:55.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:55 smithi172 ceph-mon[32456]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-23T18:18:55.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:55 smithi173 ceph-mon[32906]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-23T18:18:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:56.429 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:56.430 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:57.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:57 smithi037 ceph-mon[29404]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-23T18:18:57.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:57 smithi172 ceph-mon[32456]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-23T18:18:57.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:57 smithi173 ceph-mon[32906]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-23T18:18:59.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:18:59 smithi172 ceph-mon[32456]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-23T18:18:59.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:18:59 smithi173 ceph-mon[32906]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-23T18:18:59.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:18:59 smithi037 ceph-mon[29404]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-23T18:19:01.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:01 smithi172 ceph-mon[32456]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-23T18:19:01.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:01 smithi173 ceph-mon[32906]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-23T18:19:01.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:01 smithi037 ceph-mon[29404]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-23T18:19:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 -- 172.21.15.37:0/1530894163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8102ac0 msgr2=0x7f2ec8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 --2- 172.21.15.37:0/1530894163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8102ac0 0x7f2ec8102f40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009990 tx=0x7f2ebc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 -- 172.21.15.37:0/1530894163 shutdown_connections 2024-04-23T18:19:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 --2- 172.21.15.37:0/1530894163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ec8103480 0x7f2ec810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 --2- 172.21.15.37:0/1530894163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ec8108ab0 0x7f2ec8108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 --2- 172.21.15.37:0/1530894163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8102ac0 0x7f2ec8102f40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 -- 172.21.15.37:0/1530894163 >> 172.21.15.37:0/1530894163 conn(0x7f2ec8079bf0 msgr2=0x7f2ec807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 -- 172.21.15.37:0/1530894163 shutdown_connections 2024-04-23T18:19:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.378+0000 7f2ece637700 1 -- 172.21.15.37:0/1530894163 wait complete. 2024-04-23T18:19:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 Processor -- start 2024-04-23T18:19:03.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 -- start start 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ec8102ac0 0x7f2ec8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ec8103480 0x7f2ec81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8108ab0 0x7f2ec819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8112cf0 con 0x7f2ec8108ab0 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8112b70 con 0x7f2ec8102ac0 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2ece637700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8112e70 con 0x7f2ec8103480 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2eccbd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8108ab0 0x7f2ec819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2eccbd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8108ab0 0x7f2ec819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36884/0 (socket says 172.21.15.37:36884) 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.379+0000 7f2eccbd4700 1 -- 172.21.15.37:0/3798683979 learned_addr learned my addr 172.21.15.37:0/3798683979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ec77fe700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ec8103480 0x7f2ec81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ec7fff700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ec8102ac0 0x7f2ec8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2eccbd4700 1 -- 172.21.15.37:0/3798683979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ec8103480 msgr2=0x7f2ec81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2eccbd4700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ec8103480 0x7f2ec81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2eccbd4700 1 -- 172.21.15.37:0/3798683979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ec8102ac0 msgr2=0x7f2ec8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2eccbd4700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ec8102ac0 0x7f2ec8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2eccbd4700 1 -- 172.21.15.37:0/3798683979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec819e190 con 0x7f2ec8108ab0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2eccbd4700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8108ab0 0x7f2ec819da50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000ada0 tx=0x7f2ec000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0017020 con 0x7f2ec8108ab0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec819e480 con 0x7f2ec8108ab0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec000ce30 con 0x7f2ec8108ab0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0005350 con 0x7f2ec8108ab0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.380+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec8074c70 con 0x7f2ec8108ab0 2024-04-23T18:19:03.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.381+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ec8063150 con 0x7f2ec8108ab0 2024-04-23T18:19:03.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.382+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ec000c980 con 0x7f2ec8108ab0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec57fa700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2eb00a3cd0 0x7f2eb00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec7fff700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2eb00a3cd0 0x7f2eb00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2ec00d0500 con 0x7f2ec8108ab0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec57fa700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] conn(0x7f2eb00ad4b0 0x7f2eb00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 --> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ec000fe50 con 0x7f2eb00ad4b0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2ec00d09e0 con 0x7f2ec8108ab0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.383+0000 7f2ec77fe700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] conn(0x7f2eb00ad4b0 0x7f2eb00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.384+0000 7f2ec7fff700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2eb00a3cd0 0x7f2eb00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005950 tx=0x7f2eb80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.384+0000 7f2ec77fe700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] conn(0x7f2eb00ad4b0 0x7f2eb00af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.385+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== osd.3 v2:172.21.15.37:6810/1815509049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ec000fe50 con 0x7f2eb00ad4b0 2024-04-23T18:19:03.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.401+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 --> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ec80635d0 con 0x7f2eb00ad4b0 2024-04-23T18:19:03.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.401+0000 7f2ec57fa700 1 -- 172.21.15.37:0/3798683979 <== osd.3 v2:172.21.15.37:6810/1815509049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ec80635d0 con 0x7f2eb00ad4b0 2024-04-23T18:19:03.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] conn(0x7f2eb00ad4b0 msgr2=0x7f2eb00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] conn(0x7f2eb00ad4b0 0x7f2eb00af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2eb00a3cd0 msgr2=0x7f2eb00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2eb00a3cd0 0x7f2eb00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005950 tx=0x7f2eb80058e0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8108ab0 msgr2=0x7f2ec819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 --2- 172.21.15.37:0/3798683979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ec8108ab0 0x7f2ec819da50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000ada0 tx=0x7f2ec000c330 comp rx=0 tx=0).stop 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2eccbd4700 1 -- 172.21.15.37:0/3798683979 reap_dead start 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 shutdown_connections 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 >> 172.21.15.37:0/3798683979 conn(0x7f2ec8079bf0 msgr2=0x7f2ec8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 shutdown_connections 2024-04-23T18:19:03.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.402+0000 7f2ece637700 1 -- 172.21.15.37:0/3798683979 wait complete. 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 -- 172.21.15.37:0/876899054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938102300 msgr2=0x7f093810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 --2- 172.21.15.37:0/876899054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938102300 0x7f093810e7f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f093000b0a0 tx=0x7f093000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 -- 172.21.15.37:0/876899054 shutdown_connections 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 --2- 172.21.15.37:0/876899054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938102300 0x7f093810e7f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 --2- 172.21.15.37:0/876899054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0938107990 0x7f0938107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 --2- 172.21.15.37:0/876899054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0938101940 0x7f0938101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 -- 172.21.15.37:0/876899054 >> 172.21.15.37:0/876899054 conn(0x7f09380788f0 msgr2=0x7f0938078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 -- 172.21.15.37:0/876899054 shutdown_connections 2024-04-23T18:19:03.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.578+0000 7f093e6f6700 1 -- 172.21.15.37:0/876899054 wait complete. 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 Processor -- start 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 -- start start 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938101940 0x7f09381034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0938102300 0x7f0938103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0938107990 0x7f0938103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0938111af0 con 0x7f0938101940 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0938111970 con 0x7f0938107990 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f093e6f6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0938111c70 con 0x7f0938102300 2024-04-23T18:19:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f0937fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938101940 0x7f09381034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.579+0000 7f0937fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938101940 0x7f09381034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36900/0 (socket says 172.21.15.37:36900) 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 -- 172.21.15.37:0/971013143 learned_addr learned my addr 172.21.15.37:0/971013143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f09377fe700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0938102300 0x7f0938103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 -- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0938102300 msgr2=0x7f0938103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0938102300 0x7f0938103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f093cc93700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0938107990 0x7f0938103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 -- 172.21.15.37:0/971013143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0938107990 msgr2=0x7f0938103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0938107990 0x7f0938103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 -- 172.21.15.37:0/971013143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09381a75c0 con 0x7f0938101940 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f0937fff700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938101940 0x7f09381034f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f092800d950 tx=0x7f092800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928016070 con 0x7f0938101940 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09381a78b0 con 0x7f0938101940 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0928010460 con 0x7f0938101940 2024-04-23T18:19:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092800f620 con 0x7f0938101940 2024-04-23T18:19:03.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.580+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09381a7e40 con 0x7f0938101940 2024-04-23T18:19:03.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.581+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0938063110 con 0x7f0938101940 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.582+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0928019070 con 0x7f0938101940 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09357fa700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f09200a3cd0 0x7f09200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09377fe700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f09200a3cd0 0x7f09200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f09280d0540 con 0x7f0938101940 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09357fa700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] conn(0x7f09200ad4b0 0x7f09200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 --> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0928014e80 con 0x7f09200ad4b0 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f09280d08f0 con 0x7f0938101940 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.583+0000 7f09377fe700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f09200a3cd0 0x7f09200a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0938079030 tx=0x7f092c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:03.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.584+0000 7f093cc93700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] conn(0x7f09200ad4b0 0x7f09200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:03.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.584+0000 7f093cc93700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] conn(0x7f09200ad4b0 0x7f09200af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:03.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.585+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== osd.2 v2:172.21.15.173:6800/4145857941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0928014e80 con 0x7f09200ad4b0 2024-04-23T18:19:03.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.601+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 --> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0938063510 con 0x7f09200ad4b0 2024-04-23T18:19:03.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f09357fa700 1 -- 172.21.15.37:0/971013143 <== osd.2 v2:172.21.15.173:6800/4145857941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0938063510 con 0x7f09200ad4b0 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] conn(0x7f09200ad4b0 msgr2=0x7f09200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093e6f6700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] conn(0x7f09200ad4b0 0x7f09200af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f09200a3cd0 msgr2=0x7f09200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093e6f6700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f09200a3cd0 0x7f09200a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0938079030 tx=0x7f092c009260 comp rx=0 tx=0).stop 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938101940 msgr2=0x7f09381034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093e6f6700 1 --2- 172.21.15.37:0/971013143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0938101940 0x7f09381034f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f092800d950 tx=0x7f092800dd10 comp rx=0 tx=0).stop 2024-04-23T18:19:03.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.602+0000 7f093cc93700 1 -- 172.21.15.37:0/971013143 reap_dead start 2024-04-23T18:19:03.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.603+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 shutdown_connections 2024-04-23T18:19:03.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.603+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 >> 172.21.15.37:0/971013143 conn(0x7f09380788f0 msgr2=0x7f093806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:03.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.603+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 shutdown_connections 2024-04-23T18:19:03.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:03.603+0000 7f093e6f6700 1 -- 172.21.15.37:0/971013143 wait complete. 2024-04-23T18:19:03.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:03 smithi172 ceph-mon[32456]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-23T18:19:03.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:03 smithi173 ceph-mon[32906]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-23T18:19:03.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:03 smithi037 ceph-mon[29404]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-23T18:19:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 -- 172.21.15.37:0/3398763546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70010f490 msgr2=0x7fe700111880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 --2- 172.21.15.37:0/3398763546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70010f490 0x7fe700111880 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800b0a0 tx=0x7fe6f800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 -- 172.21.15.37:0/3398763546 shutdown_connections 2024-04-23T18:19:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 --2- 172.21.15.37:0/3398763546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70010f490 0x7fe700111880 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 --2- 172.21.15.37:0/3398763546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70006b1f0 0x7fe70006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 --2- 172.21.15.37:0/3398763546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe70006bb10 0x7fe70010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 -- 172.21.15.37:0/3398763546 >> 172.21.15.37:0/3398763546 conn(0x7fe7000789e0 msgr2=0x7fe700078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 -- 172.21.15.37:0/3398763546 shutdown_connections 2024-04-23T18:19:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.316+0000 7fe70703d700 1 -- 172.21.15.37:0/3398763546 wait complete. 2024-04-23T18:19:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 Processor -- start 2024-04-23T18:19:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 -- start start 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe70006b1f0 0x7fe70019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70006bb10 0x7fe70019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70010f490 0x7fe70019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe700114e40 con 0x7fe70006bb10 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe700114cc0 con 0x7fe70006b1f0 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe70703d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe700114fc0 con 0x7fe70010f490 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70006bb10 0x7fe70019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70006bb10 0x7fe70019b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36920/0 (socket says 172.21.15.37:36920) 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe6fffff700 1 -- 172.21.15.37:0/2470418980 learned_addr learned my addr 172.21.15.37:0/2470418980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe704dd9700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe70006b1f0 0x7fe70019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.317+0000 7fe7055da700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70010f490 0x7fe70019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fffff700 1 -- 172.21.15.37:0/2470418980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70010f490 msgr2=0x7fe70019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fffff700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70010f490 0x7fe70019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fffff700 1 -- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe70006b1f0 msgr2=0x7fe70019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fffff700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe70006b1f0 0x7fe70019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fffff700 1 -- 172.21.15.37:0/2470418980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7001a02d0 con 0x7fe70006bb10 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fffff700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70006bb10 0x7fe70019b850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00d950 tx=0x7fe6ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec016070 con 0x7fe70006bb10 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ec009df0 con 0x7fe70006bb10 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7001a05c0 con 0x7fe70006bb10 2024-04-23T18:19:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00f3c0 con 0x7fe70006bb10 2024-04-23T18:19:04.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.318+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7001a7e40 con 0x7fe70006bb10 2024-04-23T18:19:04.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.319+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe700063190 con 0x7fe70006bb10 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.320+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6ec019070 con 0x7fe70006bb10 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe6fdffb700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe6f00a3cd0 0x7fe6f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe6ec0d0450 con 0x7fe70006bb10 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe704dd9700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe6f00a3cd0 0x7fe6f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe6fdffb700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] conn(0x7fe6f00ad4b0 0x7fe6f00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 --> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6ec013ec0 con 0x7fe6f00ad4b0 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe6ec0d0930 con 0x7fe70006bb10 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe7055da700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] conn(0x7fe6f00ad4b0 0x7fe6f00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.321+0000 7fe704dd9700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe6f00a3cd0 0x7fe6f00a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4005950 tx=0x7fe6f400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:04.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.322+0000 7fe7055da700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] conn(0x7fe6f00ad4b0 0x7fe6f00af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:04.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.323+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== osd.10 v2:172.21.15.172:6824/1398661350 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6ec013ec0 con 0x7fe6f00ad4b0 2024-04-23T18:19:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.339+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 --> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe700063610 con 0x7fe6f00ad4b0 2024-04-23T18:19:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe6fdffb700 1 -- 172.21.15.37:0/2470418980 <== osd.10 v2:172.21.15.172:6824/1398661350 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe700063610 con 0x7fe6f00ad4b0 2024-04-23T18:19:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] conn(0x7fe6f00ad4b0 msgr2=0x7fe6f00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] conn(0x7fe6f00ad4b0 0x7fe6f00af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe6f00a3cd0 msgr2=0x7fe6f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fe6f00a3cd0 0x7fe6f00a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4005950 tx=0x7fe6f400a300 comp rx=0 tx=0).stop 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70006bb10 msgr2=0x7fe70019b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 --2- 172.21.15.37:0/2470418980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe70006bb10 0x7fe70019b850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00d950 tx=0x7fe6ec00dc60 comp rx=0 tx=0).stop 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe7055da700 1 -- 172.21.15.37:0/2470418980 reap_dead start 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 shutdown_connections 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 >> 172.21.15.37:0/2470418980 conn(0x7fe7000789e0 msgr2=0x7fe7001108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 shutdown_connections 2024-04-23T18:19:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:04.340+0000 7fe70703d700 1 -- 172.21.15.37:0/2470418980 wait complete. 2024-04-23T18:19:05.198 INFO:teuthology.orchestra.run.smithi037.stdout:322122547224 2024-04-23T18:19:05.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-23T18:19:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 -- 172.21.15.37:0/195763518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 msgr2=0x7fcd3410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/195763518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd3410f900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b0a0 tx=0x7fcd2c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 -- 172.21.15.37:0/195763518 shutdown_connections 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/195763518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd3410f900 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/195763518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd34108a30 0x7fcd34108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/195763518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd3406a750 0x7fcd3406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 -- 172.21.15.37:0/195763518 >> 172.21.15.37:0/195763518 conn(0x7fcd340788f0 msgr2=0x7fcd34078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 -- 172.21.15.37:0/195763518 shutdown_connections 2024-04-23T18:19:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.420+0000 7fcd38e3b700 1 -- 172.21.15.37:0/195763518 wait complete. 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.421+0000 7fcd38e3b700 1 Processor -- start 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 -- start start 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3406a750 0x7fcd34199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd34199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd34108a30 0x7fcd3419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34112bd0 con 0x7fcd3406b0f0 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34112a50 con 0x7fcd3406a750 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd38e3b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34112d50 con 0x7fcd34108a30 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd3259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd34199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd3259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd34199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52168/0 (socket says 172.21.15.37:52168) 2024-04-23T18:19:05.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.422+0000 7fcd3259c700 1 -- 172.21.15.37:0/3347478732 learned_addr learned my addr 172.21.15.37:0/3347478732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd32d9d700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd34108a30 0x7fcd3419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd2a59c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3406a750 0x7fcd34199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd3259c700 1 -- 172.21.15.37:0/3347478732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd34108a30 msgr2=0x7fcd3419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd3259c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd34108a30 0x7fcd3419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd3259c700 1 -- 172.21.15.37:0/3347478732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3406a750 msgr2=0x7fcd34199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd3259c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3406a750 0x7fcd34199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd3259c700 1 -- 172.21.15.37:0/3347478732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3419e160 con 0x7fcd3406b0f0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd2a59c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3406a750 0x7fcd34199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd3259c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd34199690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c920 tx=0x7fcd1c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c007b00 con 0x7fcd3406b0f0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3419e450 con 0x7fcd3406b0f0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd341a5cd0 con 0x7fcd3406b0f0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd1c004ce0 con 0x7fcd3406b0f0 2024-04-23T18:19:05.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.423+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c005650 con 0x7fcd3406b0f0 2024-04-23T18:19:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.424+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd34063150 con 0x7fcd3406b0f0 2024-04-23T18:19:05.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.426+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcd1c007ca0 con 0x7fcd3406b0f0 2024-04-23T18:19:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.426+0000 7fcd2bfff700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcd200a3cd0 0x7fcd200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.426+0000 7fcd2a59c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcd200a3cd0 0x7fcd200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fcd1c0d04b0 con 0x7fcd3406b0f0 2024-04-23T18:19:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd2bfff700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] conn(0x7fcd200ad4b0 0x7fcd200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 --> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd200aff80 con 0x7fcd200ad4b0 2024-04-23T18:19:05.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fcd1c0d0990 con 0x7fcd3406b0f0 2024-04-23T18:19:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd32d9d700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] conn(0x7fcd200ad4b0 0x7fcd200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd2a59c700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcd200a3cd0 0x7fcd200a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005fd0 tx=0x7fcd24005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.427+0000 7fcd32d9d700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] conn(0x7fcd200ad4b0 0x7fcd200af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.428+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== osd.0 v2:172.21.15.37:6802/3727253948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd200aff80 con 0x7fcd200ad4b0 2024-04-23T18:19:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.445+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 --> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd34063580 con 0x7fcd200ad4b0 2024-04-23T18:19:05.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.445+0000 7fcd2bfff700 1 -- 172.21.15.37:0/3347478732 <== osd.0 v2:172.21.15.37:6802/3727253948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd34063580 con 0x7fcd200ad4b0 2024-04-23T18:19:05.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] conn(0x7fcd200ad4b0 msgr2=0x7fcd200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:05.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] conn(0x7fcd200ad4b0 0x7fcd200af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcd200a3cd0 msgr2=0x7fcd200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fcd200a3cd0 0x7fcd200a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005fd0 tx=0x7fcd24005db0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 msgr2=0x7fcd34199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 --2- 172.21.15.37:0/3347478732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3406b0f0 0x7fcd34199690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c920 tx=0x7fcd1c00cce0 comp rx=0 tx=0).stop 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd32d9d700 1 -- 172.21.15.37:0/3347478732 reap_dead start 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 shutdown_connections 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 >> 172.21.15.37:0/3347478732 conn(0x7fcd340788f0 msgr2=0x7fcd340ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 shutdown_connections 2024-04-23T18:19:05.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:05.446+0000 7fcd38e3b700 1 -- 172.21.15.37:0/3347478732 wait complete. 2024-04-23T18:19:05.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:05 smithi172 ceph-mon[32456]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T18:19:05.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:05 smithi173 ceph-mon[32906]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T18:19:05.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:05 smithi037 ceph-mon[29404]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T18:19:06.237 INFO:teuthology.orchestra.run.smithi037.stdout:34359738441 2024-04-23T18:19:06.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T18:19:07.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:07 smithi037 ceph-mon[29404]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-23T18:19:07.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:07 smithi172 ceph-mon[32456]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-23T18:19:07.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:07 smithi173 ceph-mon[32906]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-23T18:19:08.490 INFO:teuthology.orchestra.run.smithi037.stdout:107374182457 2024-04-23T18:19:08.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T18:19:09.355 INFO:teuthology.orchestra.run.smithi037.stdout:77309411391 2024-04-23T18:19:09.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 -- 172.21.15.37:0/4043378445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc428108a60 msgr2=0x7fc428108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 --2- 172.21.15.37:0/4043378445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc428108a60 0x7fc428108e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc418009990 tx=0x7fc418009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 -- 172.21.15.37:0/4043378445 shutdown_connections 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 --2- 172.21.15.37:0/4043378445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc428108a60 0x7fc428108e40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 --2- 172.21.15.37:0/4043378445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc42806bbb0 0x7fc42810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 --2- 172.21.15.37:0/4043378445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc42806b1f0 0x7fc42806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 -- 172.21.15.37:0/4043378445 >> 172.21.15.37:0/4043378445 conn(0x7fc4280789e0 msgr2=0x7fc428078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 -- 172.21.15.37:0/4043378445 shutdown_connections 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.113+0000 7fc42ccae700 1 -- 172.21.15.37:0/4043378445 wait complete. 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 Processor -- start 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 -- start start 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc42806b1f0 0x7fc428199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc42806bbb0 0x7fc4281995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc428108a60 0x7fc42819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112c50 con 0x7fc42806bbb0 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112ad0 con 0x7fc428108a60 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42ccae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112dd0 con 0x7fc42806b1f0 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc425d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc42806bbb0 0x7fc4281995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc425d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc42806bbb0 0x7fc4281995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52178/0 (socket says 172.21.15.37:52178) 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc425d9b700 1 -- 172.21.15.37:0/2912548105 learned_addr learned my addr 172.21.15.37:0/2912548105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.114+0000 7fc42659c700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc42806b1f0 0x7fc428199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc425d9b700 1 -- 172.21.15.37:0/2912548105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc42806b1f0 msgr2=0x7fc428199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc426d9d700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc428108a60 0x7fc42819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc425d9b700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc42806b1f0 0x7fc428199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc425d9b700 1 -- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc428108a60 msgr2=0x7fc42819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc425d9b700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc428108a60 0x7fc42819d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc425d9b700 1 -- 172.21.15.37:0/2912548105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42819e070 con 0x7fc42806bbb0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc42659c700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc42806b1f0 0x7fc428199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc425d9b700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc42806bbb0 0x7fc4281995a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c9b0 tx=0x7fc41c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c007b00 con 0x7fc42806bbb0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc41c007ca0 con 0x7fc42806bbb0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42819e360 con 0x7fc42806bbb0 2024-04-23T18:19:09.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c018730 con 0x7fc42806bbb0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.115+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4281a5be0 con 0x7fc42806bbb0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.116+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4280631d0 con 0x7fc42806bbb0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.117+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc41c00f450 con 0x7fc42806bbb0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc4177fe700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4100a3cd0 0x7fc4100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc42659c700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4100a3cd0 0x7fc4100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc41c0d08f0 con 0x7fc42806bbb0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc4177fe700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] conn(0x7fc4100ad4b0 0x7fc4100af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 --> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc41c004500 con 0x7fc4100ad4b0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc41c0d0dd0 con 0x7fc42806bbb0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc42659c700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4100a3cd0 0x7fc4100a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc418006010 tx=0x7fc41800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.118+0000 7fc426d9d700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] conn(0x7fc4100ad4b0 0x7fc4100af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.119+0000 7fc426d9d700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] conn(0x7fc4100ad4b0 0x7fc4100af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:09.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.120+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== osd.7 v2:172.21.15.172:6816/3090366504 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc41c004500 con 0x7fc4100ad4b0 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.136+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 --> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc428063650 con 0x7fc4100ad4b0 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.136+0000 7fc4177fe700 1 -- 172.21.15.37:0/2912548105 <== osd.7 v2:172.21.15.172:6816/3090366504 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc428063650 con 0x7fc4100ad4b0 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] conn(0x7fc4100ad4b0 msgr2=0x7fc4100af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] conn(0x7fc4100ad4b0 0x7fc4100af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4100a3cd0 msgr2=0x7fc4100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4100a3cd0 0x7fc4100a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc418006010 tx=0x7fc41800b540 comp rx=0 tx=0).stop 2024-04-23T18:19:09.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc42806bbb0 msgr2=0x7fc4281995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 --2- 172.21.15.37:0/2912548105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc42806bbb0 0x7fc4281995a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c9b0 tx=0x7fc41c00cd70 comp rx=0 tx=0).stop 2024-04-23T18:19:09.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc426d9d700 1 -- 172.21.15.37:0/2912548105 reap_dead start 2024-04-23T18:19:09.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 shutdown_connections 2024-04-23T18:19:09.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 >> 172.21.15.37:0/2912548105 conn(0x7fc4280789e0 msgr2=0x7fc4280ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:09.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 shutdown_connections 2024-04-23T18:19:09.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.137+0000 7fc42ccae700 1 -- 172.21.15.37:0/2912548105 wait complete. 2024-04-23T18:19:09.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:09 smithi037 ceph-mon[29404]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T18:19:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:09.831 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:09.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:09 smithi172 ceph-mon[32456]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T18:19:09.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:09 smithi173 ceph-mon[32906]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T18:19:09.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/2675200542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0102300 msgr2=0x7f5ce010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/2675200542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0102300 0x7f5ce010e7f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800b0a0 tx=0x7f5cd800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/2675200542 shutdown_connections 2024-04-23T18:19:09.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/2675200542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0102300 0x7f5ce010e7f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/2675200542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0107990 0x7f5ce0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/2675200542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce0101940 0x7f5ce0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/2675200542 >> 172.21.15.37:0/2675200542 conn(0x7f5ce00788f0 msgr2=0x7f5ce0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:09.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/2675200542 shutdown_connections 2024-04-23T18:19:09.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/2675200542 wait complete. 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.869+0000 7f5ce6dd8700 1 Processor -- start 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 -- start start 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0101940 0x7f5ce01034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce0102300 0x7f5ce0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0107990 0x7f5ce0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0111b10 con 0x7f5ce0101940 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0111990 con 0x7f5ce0102300 2024-04-23T18:19:09.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce6dd8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0111c90 con 0x7f5ce0107990 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0101940 0x7f5ce01034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0101940 0x7f5ce01034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52192/0 (socket says 172.21.15.37:52192) 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 -- 172.21.15.37:0/349135700 learned_addr learned my addr 172.21.15.37:0/349135700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5cdffff700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce0102300 0x7f5ce0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce5375700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0107990 0x7f5ce0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 -- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0107990 msgr2=0x7f5ce0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce0107990 0x7f5ce0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 -- 172.21.15.37:0/349135700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce0102300 msgr2=0x7f5ce0103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce0102300 0x7f5ce0103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 -- 172.21.15.37:0/349135700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce01a7620 con 0x7f5ce0101940 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5cdffff700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce0102300 0x7f5ce0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.870+0000 7f5ce4b74700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0101940 0x7f5ce01034f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c970 tx=0x7f5cd400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.871+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40078d0 con 0x7f5ce0101940 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.871+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd4018480 con 0x7f5ce0101940 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.871+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400f6b0 con 0x7f5ce0101940 2024-04-23T18:19:09.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.871+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01a7910 con 0x7f5ce0101940 2024-04-23T18:19:09.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.872+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a7dd0 con 0x7f5ce0101940 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.872+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ce0063110 con 0x7f5ce0101940 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.873+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5cd4007a70 con 0x7f5ce0101940 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.874+0000 7f5cddffb700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd00a3c30 0x7f5cd00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.874+0000 7f5cdffff700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd00a3c30 0x7f5cd00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.874+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5cd40d0860 con 0x7f5ce0101940 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.874+0000 7f5cddffb700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] conn(0x7f5cd00ad410 0x7f5cd00af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.874+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 --> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5cd4016e50 con 0x7f5cd00ad410 2024-04-23T18:19:09.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.874+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f5cd40d0c10 con 0x7f5ce0101940 2024-04-23T18:19:09.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.875+0000 7f5cdffff700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd00a3c30 0x7f5cd00a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0079040 tx=0x7f5ccc00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:09.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.875+0000 7f5ce5375700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] conn(0x7f5cd00ad410 0x7f5cd00af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:09.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.875+0000 7f5ce5375700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] conn(0x7f5cd00ad410 0x7f5cd00af830 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:09.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.876+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== osd.8 v2:172.21.15.173:6816/1969719555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cd4016e50 con 0x7f5cd00ad410 2024-04-23T18:19:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.892+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 --> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ce0063590 con 0x7f5cd00ad410 2024-04-23T18:19:09.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5cddffb700 1 -- 172.21.15.37:0/349135700 <== osd.8 v2:172.21.15.173:6816/1969719555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ce0063590 con 0x7f5cd00ad410 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] conn(0x7f5cd00ad410 msgr2=0x7f5cd00af830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] conn(0x7f5cd00ad410 0x7f5cd00af830 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd00a3c30 msgr2=0x7f5cd00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5cd00a3c30 0x7f5cd00a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0079040 tx=0x7f5ccc00b3c0 comp rx=0 tx=0).stop 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0101940 msgr2=0x7f5ce01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce6dd8700 1 --2- 172.21.15.37:0/349135700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ce0101940 0x7f5ce01034f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c970 tx=0x7f5cd400cc80 comp rx=0 tx=0).stop 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.893+0000 7f5ce5375700 1 -- 172.21.15.37:0/349135700 reap_dead start 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.894+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 shutdown_connections 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.894+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 >> 172.21.15.37:0/349135700 conn(0x7f5ce00788f0 msgr2=0x7f5ce006aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.894+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 shutdown_connections 2024-04-23T18:19:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:09.894+0000 7f5ce6dd8700 1 -- 172.21.15.37:0/349135700 wait complete. 2024-04-23T18:19:10.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 -- 172.21.15.37:0/2227462334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 msgr2=0x7f62d0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:10.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 --2- 172.21.15.37:0/2227462334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0108e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009990 tx=0x7f62c0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 -- 172.21.15.37:0/2227462334 shutdown_connections 2024-04-23T18:19:10.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 --2- 172.21.15.37:0/2227462334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0108e10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 --2- 172.21.15.37:0/2227462334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62d006b0f0 0x7f62d010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 --2- 172.21.15.37:0/2227462334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62d006a750 0x7f62d006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 -- 172.21.15.37:0/2227462334 >> 172.21.15.37:0/2227462334 conn(0x7f62d00788f0 msgr2=0x7f62d0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 -- 172.21.15.37:0/2227462334 shutdown_connections 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.362+0000 7f62d5fee700 1 -- 172.21.15.37:0/2227462334 wait complete. 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 Processor -- start 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 -- start start 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62d006a750 0x7f62d0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62d006b0f0 0x7f62d0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0112c80 con 0x7f62d0108a30 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0112b00 con 0x7f62d006b0f0 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62d5fee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0112e00 con 0x7f62d006a750 2024-04-23T18:19:10.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62cffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62cffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52202/0 (socket says 172.21.15.37:52202) 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62cffff700 1 -- 172.21.15.37:0/3577925188 learned_addr learned my addr 172.21.15.37:0/3577925188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.363+0000 7f62cf7fe700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62d006a750 0x7f62d0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62ceffd700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62d006b0f0 0x7f62d0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62cffff700 1 -- 172.21.15.37:0/3577925188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62d006a750 msgr2=0x7f62d0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62cffff700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62d006a750 0x7f62d0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62cffff700 1 -- 172.21.15.37:0/3577925188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62d006b0f0 msgr2=0x7f62d0101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62cffff700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62d006b0f0 0x7f62d0101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62cffff700 1 -- 172.21.15.37:0/3577925188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d01a7740 con 0x7f62d0108a30 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62ceffd700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62d006b0f0 0x7f62d0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:10.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62cffff700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0102490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f62c800ada0 tx=0x7f62c800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:10.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8017020 con 0x7f62d0108a30 2024-04-23T18:19:10.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c800ce30 con 0x7f62d0108a30 2024-04-23T18:19:10.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d01a7a30 con 0x7f62d0108a30 2024-04-23T18:19:10.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8005350 con 0x7f62d0108a30 2024-04-23T18:19:10.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.364+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d01a7f20 con 0x7f62d0108a30 2024-04-23T18:19:10.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.365+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62d0063110 con 0x7f62d0108a30 2024-04-23T18:19:10.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.366+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f62c800c980 con 0x7f62d0108a30 2024-04-23T18:19:10.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62ccff9700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f62b80a3cd0 0x7f62b80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62cf7fe700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f62b80a3cd0 0x7f62b80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f62c80d0500 con 0x7f62d0108a30 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62ccff9700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] conn(0x7f62b80ad4b0 0x7f62b80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 --> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62c800fe50 con 0x7f62b80ad4b0 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f62c80d09e0 con 0x7f62d0108a30 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62ceffd700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] conn(0x7f62b80ad4b0 0x7f62b80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.367+0000 7f62cf7fe700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f62b80a3cd0 0x7f62b80a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009750 tx=0x7f62c0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.368+0000 7f62ceffd700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] conn(0x7f62b80ad4b0 0x7f62b80af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:10.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.368+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== osd.9 v2:172.21.15.37:6826/1483571052 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62c800fe50 con 0x7f62b80ad4b0 2024-04-23T18:19:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.385+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 --> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62d0063590 con 0x7f62b80ad4b0 2024-04-23T18:19:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.385+0000 7f62ccff9700 1 -- 172.21.15.37:0/3577925188 <== osd.9 v2:172.21.15.37:6826/1483571052 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f62d0063590 con 0x7f62b80ad4b0 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.385+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] conn(0x7f62b80ad4b0 msgr2=0x7f62b80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.385+0000 7f62d5fee700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] conn(0x7f62b80ad4b0 0x7f62b80af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f62b80a3cd0 msgr2=0x7f62b80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f62b80a3cd0 0x7f62b80a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009750 tx=0x7f62c0005fd0 comp rx=0 tx=0).stop 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 msgr2=0x7f62d0102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 --2- 172.21.15.37:0/3577925188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62d0108a30 0x7f62d0102490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f62c800ada0 tx=0x7f62c800c330 comp rx=0 tx=0).stop 2024-04-23T18:19:10.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62cffff700 1 -- 172.21.15.37:0/3577925188 reap_dead start 2024-04-23T18:19:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 shutdown_connections 2024-04-23T18:19:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 >> 172.21.15.37:0/3577925188 conn(0x7f62d00788f0 msgr2=0x7f62d00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 shutdown_connections 2024-04-23T18:19:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:10.386+0000 7f62d5fee700 1 -- 172.21.15.37:0/3577925188 wait complete. 2024-04-23T18:19:10.911 INFO:teuthology.orchestra.run.smithi037.stdout:261993005091 2024-04-23T18:19:10.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-23T18:19:11.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:11 smithi172 ceph-mon[32456]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:19:11.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:11 smithi173 ceph-mon[32906]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:19:11.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:11 smithi037 ceph-mon[29404]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:19:12.528 INFO:teuthology.orchestra.run.smithi037.stdout:296352743455 2024-04-23T18:19:12.529 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 -- 172.21.15.37:0/1863662802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c108a30 msgr2=0x7f441c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 --2- 172.21.15.37:0/1863662802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c108a30 0x7f441c108e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f440c00b0a0 tx=0x7f440c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 -- 172.21.15.37:0/1863662802 shutdown_connections 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 --2- 172.21.15.37:0/1863662802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c108a30 0x7f441c108e10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 --2- 172.21.15.37:0/1863662802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c06b0f0 0x7f441c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 --2- 172.21.15.37:0/1863662802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c06a750 0x7f441c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 -- 172.21.15.37:0/1863662802 >> 172.21.15.37:0/1863662802 conn(0x7f441c0788f0 msgr2=0x7f441c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 -- 172.21.15.37:0/1863662802 shutdown_connections 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.214+0000 7f4420dca700 1 -- 172.21.15.37:0/1863662802 wait complete. 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 Processor -- start 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 -- start start 2024-04-23T18:19:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c06a750 0x7f441c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c06b0f0 0x7f441c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c108a30 0x7f441c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f441c112bb0 con 0x7f441c06a750 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f441c112a30 con 0x7f441c06b0f0 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f4420dca700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f441c112d30 con 0x7f441c108a30 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f441a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c06a750 0x7f441c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f441a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c06a750 0x7f441c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52232/0 (socket says 172.21.15.37:52232) 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.215+0000 7f441a59c700 1 -- 172.21.15.37:0/1409715364 learned_addr learned my addr 172.21.15.37:0/1409715364 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441ad9d700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c108a30 0x7f441c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f4419d9b700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c06b0f0 0x7f441c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441a59c700 1 -- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c108a30 msgr2=0x7f441c19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441a59c700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c108a30 0x7f441c19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441a59c700 1 -- 172.21.15.37:0/1409715364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c06b0f0 msgr2=0x7f441c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441a59c700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c06b0f0 0x7f441c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441a59c700 1 -- 172.21.15.37:0/1409715364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c19e090 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f4419d9b700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441c06b0f0 0x7f441c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f441a59c700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c06a750 0x7f441c199080 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f440c00b070 tx=0x7f440c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c021070 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c19e320 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c007c80 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c01d450 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.216+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a5c00 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.217+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f441c0631d0 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.218+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f440c004000 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.219+0000 7f44137fe700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f44080a3cd0 0x7f44080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.219+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f440c00f020 con 0x7f441c06a750 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.219+0000 7f4419d9b700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f44080a3cd0 0x7f44080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.219+0000 7f44137fe700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] conn(0x7f44080ad4b0 0x7f44080af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.219+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 --> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f440c02ce50 con 0x7f44080ad4b0 2024-04-23T18:19:13.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.219+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f440c0d2060 con 0x7f441c06a750 2024-04-23T18:19:13.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.220+0000 7f4419d9b700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f44080a3cd0 0x7f44080a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4404005fd0 tx=0x7f4404005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:13.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.220+0000 7f441ad9d700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] conn(0x7f44080ad4b0 0x7f44080af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:13.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.220+0000 7f441ad9d700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] conn(0x7f44080ad4b0 0x7f44080af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:13.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.221+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== osd.11 v2:172.21.15.173:6824/1559500794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f440c02ce50 con 0x7f44080ad4b0 2024-04-23T18:19:13.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.238+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 --> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f441c0635d0 con 0x7f44080ad4b0 2024-04-23T18:19:13.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f44137fe700 1 -- 172.21.15.37:0/1409715364 <== osd.11 v2:172.21.15.173:6824/1559500794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f441c0635d0 con 0x7f44080ad4b0 2024-04-23T18:19:13.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] conn(0x7f44080ad4b0 msgr2=0x7f44080af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] conn(0x7f44080ad4b0 0x7f44080af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f44080a3cd0 msgr2=0x7f44080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f44080a3cd0 0x7f44080a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4404005fd0 tx=0x7f4404005e20 comp rx=0 tx=0).stop 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c06a750 msgr2=0x7f441c199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 --2- 172.21.15.37:0/1409715364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f441c06a750 0x7f441c199080 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f440c00b070 tx=0x7f440c004db0 comp rx=0 tx=0).stop 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f441ad9d700 1 -- 172.21.15.37:0/1409715364 reap_dead start 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 shutdown_connections 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.239+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 >> 172.21.15.37:0/1409715364 conn(0x7f441c0788f0 msgr2=0x7f441c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.240+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 shutdown_connections 2024-04-23T18:19:13.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:13.240+0000 7f4420dca700 1 -- 172.21.15.37:0/1409715364 wait complete. 2024-04-23T18:19:13.815 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:13 smithi037 ceph-mon[29404]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:13.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:13 smithi172 ceph-mon[32456]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:13.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:13 smithi173 ceph-mon[32906]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:15.689 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:15 smithi037 ceph-mon[29404]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:15.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:15 smithi172 ceph-mon[32456]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:15.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:15 smithi173 ceph-mon[32906]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:16.172 INFO:teuthology.orchestra.run.smithi037.stdout:227633266727 2024-04-23T18:19:16.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-23T18:19:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.389+0000 7f2dfcfce700 1 -- 172.21.15.37:0/2019405157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df8074c70 msgr2=0x7f2df8075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.389+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/2019405157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df8074c70 0x7f2df8075050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009990 tx=0x7f2de8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.390+0000 7f2dfcfce700 1 -- 172.21.15.37:0/2019405157 shutdown_connections 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.390+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/2019405157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df8074c70 0x7f2df8075050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.390+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/2019405157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df810f470 0x7f2df8111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.390+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/2019405157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df8075590 0x7f2df810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.390+0000 7f2dfcfce700 1 -- 172.21.15.37:0/2019405157 >> 172.21.15.37:0/2019405157 conn(0x7f2df8079bb0 msgr2=0x7f2df8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.391+0000 7f2dfcfce700 1 -- 172.21.15.37:0/2019405157 shutdown_connections 2024-04-23T18:19:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.391+0000 7f2dfcfce700 1 -- 172.21.15.37:0/2019405157 wait complete. 2024-04-23T18:19:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 Processor -- start 2024-04-23T18:19:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 -- start start 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8074c70 0x7f2df8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df8075590 0x7f2df81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df810f470 0x7f2df8103610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8114ea0 con 0x7f2df810f470 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8114d20 con 0x7f2df8074c70 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.392+0000 7f2dfcfce700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8115020 con 0x7f2df8075590 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df810f470 0x7f2df8103610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df810f470 0x7f2df8103610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55914/0 (socket says 172.21.15.37:55914) 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 -- 172.21.15.37:0/1543658819 learned_addr learned my addr 172.21.15.37:0/1543658819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 -- 172.21.15.37:0/1543658819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df8075590 msgr2=0x7f2df81030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df8075590 0x7f2df81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 -- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8074c70 msgr2=0x7f2df8102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8074c70 0x7f2df8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 -- 172.21.15.37:0/1543658819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df81ab9a0 con 0x7f2df810f470 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2df6d9d700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df810f470 0x7f2df8103610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2df000ada0 tx=0x7f2df000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.393+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0017020 con 0x7f2df810f470 2024-04-23T18:19:16.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.394+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df000ce30 con 0x7f2df810f470 2024-04-23T18:19:16.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.394+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df81abc90 con 0x7f2df810f470 2024-04-23T18:19:16.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.394+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00054a0 con 0x7f2df810f470 2024-04-23T18:19:16.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.394+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df81ac0c0 con 0x7f2df810f470 2024-04-23T18:19:16.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.396+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2df8100600 con 0x7f2df810f470 2024-04-23T18:19:16.395 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:16.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.400+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2df000c980 con 0x7f2df810f470 2024-04-23T18:19:16.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.402+0000 7f2de77fe700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2de00a3d90 0x7f2de00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:16.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.402+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2df00cfad0 con 0x7f2df810f470 2024-04-23T18:19:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.402+0000 7f2de77fe700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] conn(0x7f2de00ad570 0x7f2de00af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.402+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 --> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2df0016e70 con 0x7f2de00ad570 2024-04-23T18:19:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.403+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2df00d42a0 con 0x7f2df810f470 2024-04-23T18:19:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.403+0000 7f2df659c700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2de00a3d90 0x7f2de00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.404+0000 7f2df5d9b700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] conn(0x7f2de00ad570 0x7f2de00af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.404+0000 7f2df659c700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2de00a3d90 0x7f2de00a6250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2de8003800 tx=0x7f2de8009ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.405+0000 7f2df5d9b700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] conn(0x7f2de00ad570 0x7f2de00af990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.409+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== osd.4 v2:172.21.15.172:6808/3887503569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2df0016e70 con 0x7f2de00ad570 2024-04-23T18:19:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.442+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 --> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2df8076360 con 0x7f2de00ad570 2024-04-23T18:19:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.442+0000 7f2de77fe700 1 -- 172.21.15.37:0/1543658819 <== osd.4 v2:172.21.15.172:6808/3887503569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2df8076360 con 0x7f2de00ad570 2024-04-23T18:19:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] conn(0x7f2de00ad570 msgr2=0x7f2de00af990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] conn(0x7f2de00ad570 0x7f2de00af990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2de00a3d90 msgr2=0x7f2de00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2de00a3d90 0x7f2de00a6250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2de8003800 tx=0x7f2de8009ee0 comp rx=0 tx=0).stop 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df810f470 msgr2=0x7f2df8103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 --2- 172.21.15.37:0/1543658819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2df810f470 0x7f2df8103610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2df000ada0 tx=0x7f2df000c330 comp rx=0 tx=0).stop 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2df6d9d700 1 -- 172.21.15.37:0/1543658819 reap_dead start 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 shutdown_connections 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.443+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 >> 172.21.15.37:0/1543658819 conn(0x7f2df8079bb0 msgr2=0x7f2df80ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.444+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 shutdown_connections 2024-04-23T18:19:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:16.444+0000 7f2dfcfce700 1 -- 172.21.15.37:0/1543658819 wait complete. 2024-04-23T18:19:17.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:17 smithi037 ceph-mon[29404]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-23T18:19:17.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:17 smithi172 ceph-mon[32456]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-23T18:19:17.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:17 smithi173 ceph-mon[32906]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-23T18:19:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 -- 172.21.15.37:0/2462553419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f947006b0f0 msgr2=0x7f947010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 --2- 172.21.15.37:0/2462553419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f947006b0f0 0x7f947010f900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f946800b0a0 tx=0x7f946800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 -- 172.21.15.37:0/2462553419 shutdown_connections 2024-04-23T18:19:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 --2- 172.21.15.37:0/2462553419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f947006b0f0 0x7f947010f900 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 --2- 172.21.15.37:0/2462553419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9470108a30 0x7f9470108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 --2- 172.21.15.37:0/2462553419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947006a750 0x7f947006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 -- 172.21.15.37:0/2462553419 >> 172.21.15.37:0/2462553419 conn(0x7f94700788f0 msgr2=0x7f9470078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 -- 172.21.15.37:0/2462553419 shutdown_connections 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.891+0000 7f94748b4700 1 -- 172.21.15.37:0/2462553419 wait complete. 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 Processor -- start 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 -- start start 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947006a750 0x7f9470199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f947006b0f0 0x7f94701995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9470108a30 0x7f947019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9470112bd0 con 0x7f9470108a30 2024-04-23T18:19:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9470112a50 con 0x7f947006b0f0 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f94748b4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9470112d50 con 0x7f947006a750 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f946ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9470108a30 0x7f947019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f946ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9470108a30 0x7f947019d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55940/0 (socket says 172.21.15.37:55940) 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f946ed9d700 1 -- 172.21.15.37:0/2308040303 learned_addr learned my addr 172.21.15.37:0/2308040303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f946dd9b700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f947006b0f0 0x7f94701995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.892+0000 7f946e59c700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947006a750 0x7f9470199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946ed9d700 1 -- 172.21.15.37:0/2308040303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947006a750 msgr2=0x7f9470199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946ed9d700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947006a750 0x7f9470199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946ed9d700 1 -- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f947006b0f0 msgr2=0x7f94701995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946ed9d700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f947006b0f0 0x7f94701995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946ed9d700 1 -- 172.21.15.37:0/2308040303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947019e020 con 0x7f9470108a30 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946e59c700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f947006a750 0x7f9470199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f946ed9d700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9470108a30 0x7f947019d8e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94680062a0 tx=0x7f9468004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468021070 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9468007c80 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946801d450 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947019e2b0 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.893+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94701a5b90 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.894+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94700631d0 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9468004000 con 0x7f9470108a30 2024-04-23T18:19:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f945f7fe700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f94580a3e20 0x7f94580a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f946e59c700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f94580a3e20 0x7f94580a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f946800f020 con 0x7f9470108a30 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f945f7fe700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] conn(0x7f94580ad600 0x7f94580afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 --> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f946800aea0 con 0x7f94580ad600 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.896+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f94680d1c20 con 0x7f9470108a30 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.897+0000 7f946e59c700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f94580a3e20 0x7f94580a62e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9460005fd0 tx=0x7f9460005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.897+0000 7f946dd9b700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] conn(0x7f94580ad600 0x7f94580afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.897+0000 7f946dd9b700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] conn(0x7f94580ad600 0x7f94580afa20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.898+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== osd.1 v2:172.21.15.172:6800/1638984815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f946800aea0 con 0x7f94580ad600 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.914+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 --> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9470063650 con 0x7f94580ad600 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f945f7fe700 1 -- 172.21.15.37:0/2308040303 <== osd.1 v2:172.21.15.172:6800/1638984815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9470063650 con 0x7f94580ad600 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] conn(0x7f94580ad600 msgr2=0x7f94580afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f94748b4700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] conn(0x7f94580ad600 0x7f94580afa20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f94580a3e20 msgr2=0x7f94580a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f94748b4700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f94580a3e20 0x7f94580a62e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9460005fd0 tx=0x7f9460005d40 comp rx=0 tx=0).stop 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9470108a30 msgr2=0x7f947019d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f94748b4700 1 --2- 172.21.15.37:0/2308040303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9470108a30 0x7f947019d8e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94680062a0 tx=0x7f9468004e20 comp rx=0 tx=0).stop 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.915+0000 7f946ed9d700 1 -- 172.21.15.37:0/2308040303 reap_dead start 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.916+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 shutdown_connections 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.916+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 >> 172.21.15.37:0/2308040303 conn(0x7f94700788f0 msgr2=0x7f94700ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.916+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 shutdown_connections 2024-04-23T18:19:18.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:17.916+0000 7f94748b4700 1 -- 172.21.15.37:0/2308040303 wait complete. 2024-04-23T18:19:18.708 INFO:teuthology.orchestra.run.smithi037.stdout:55834574919 2024-04-23T18:19:18.708 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T18:19:19.132 INFO:teuthology.orchestra.run.smithi037.stdout:356482285588 2024-04-23T18:19:19.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-23T18:19:19.818 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:19 smithi037 ceph-mon[29404]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:19.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:19 smithi172 ceph-mon[32456]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:19.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:19 smithi173 ceph-mon[32906]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.928+0000 7f210b172700 1 -- 172.21.15.37:0/3784477831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 msgr2=0x7f21041118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 --2- 172.21.15.37:0/3784477831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f21041118d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b0a0 tx=0x7f20fc00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 -- 172.21.15.37:0/3784477831 shutdown_connections 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 --2- 172.21.15.37:0/3784477831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f21041118d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 --2- 172.21.15.37:0/3784477831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f210406afe0 0x7f210406b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 --2- 172.21.15.37:0/3784477831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210406b900 0x7f210410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 -- 172.21.15.37:0/3784477831 >> 172.21.15.37:0/3784477831 conn(0x7f21040788b0 msgr2=0x7f2104078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 -- 172.21.15.37:0/3784477831 shutdown_connections 2024-04-23T18:19:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 -- 172.21.15.37:0/3784477831 wait complete. 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 Processor -- start 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.929+0000 7f210b172700 1 -- start start 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210406afe0 0x7f210419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f210406b900 0x7f210419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f210419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2104114e00 con 0x7f210410f4e0 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2104114c80 con 0x7f210406afe0 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2104114f80 con 0x7f210406b900 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f210419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f210419fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55960/0 (socket says 172.21.15.37:55960) 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 -- 172.21.15.37:0/1167110666 learned_addr learned my addr 172.21.15.37:0/1167110666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f2108f0e700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210406afe0 0x7f210419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f2103fff700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f210406b900 0x7f210419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 -- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f210406b900 msgr2=0x7f210419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f210406b900 0x7f210419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 -- 172.21.15.37:0/1167110666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210406afe0 msgr2=0x7f210419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210406afe0 0x7f210419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 -- 172.21.15.37:0/1167110666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21041a02d0 con 0x7f210410f4e0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f2103fff700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f210406b900 0x7f210419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210970f700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f210419fb90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0062a0 tx=0x7f20fc004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc021070 con 0x7f210410f4e0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20fc003c00 con 0x7f210410f4e0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041a0560 con 0x7f210410f4e0 2024-04-23T18:19:19.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.930+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc004810 con 0x7f210410f4e0 2024-04-23T18:19:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.931+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a7e40 con 0x7f210410f4e0 2024-04-23T18:19:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.932+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2104063190 con 0x7f210410f4e0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.933+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f20fc003db0 con 0x7f210410f4e0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.933+0000 7f2101ffb700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20f40a3cd0 0x7f20f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2108f0e700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20f40a3cd0 0x7f20f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f20fc00f020 con 0x7f210410f4e0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2101ffb700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] conn(0x7f20f40ad4b0 0x7f20f40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 --> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20f40aff80 con 0x7f20f40ad4b0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f20fc0d1fd0 con 0x7f210410f4e0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2108f0e700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20f40a3cd0 0x7f20f40a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005950 tx=0x7f20f80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2103fff700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] conn(0x7f20f40ad4b0 0x7f20f40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.934+0000 7f2103fff700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] conn(0x7f20f40ad4b0 0x7f20f40af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.935+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== osd.5 v2:172.21.15.173:6808/1194532318 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20f40aff80 con 0x7f20f40ad4b0 2024-04-23T18:19:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.952+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 --> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2104063610 con 0x7f20f40ad4b0 2024-04-23T18:19:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.952+0000 7f2101ffb700 1 -- 172.21.15.37:0/1167110666 <== osd.5 v2:172.21.15.173:6808/1194532318 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2104063610 con 0x7f20f40ad4b0 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] conn(0x7f20f40ad4b0 msgr2=0x7f20f40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] conn(0x7f20f40ad4b0 0x7f20f40af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20f40a3cd0 msgr2=0x7f20f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f20f40a3cd0 0x7f20f40a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005950 tx=0x7f20f80058e0 comp rx=0 tx=0).stop 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 msgr2=0x7f210419fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 --2- 172.21.15.37:0/1167110666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f210410f4e0 0x7f210419fb90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0062a0 tx=0x7f20fc004d30 comp rx=0 tx=0).stop 2024-04-23T18:19:19.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210970f700 1 -- 172.21.15.37:0/1167110666 reap_dead start 2024-04-23T18:19:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 shutdown_connections 2024-04-23T18:19:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 >> 172.21.15.37:0/1167110666 conn(0x7f21040788b0 msgr2=0x7f21041109d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 shutdown_connections 2024-04-23T18:19:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:19.953+0000 7f210b172700 1 -- 172.21.15.37:0/1167110666 wait complete. 2024-04-23T18:19:20.425 INFO:teuthology.orchestra.run.smithi037.stdout:141733920825 2024-04-23T18:19:20.426 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T18:19:20.558 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 -- 172.21.15.37:0/4018189324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc102300 msgr2=0x7f53fc10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 --2- 172.21.15.37:0/4018189324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc102300 0x7f53fc10e7f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53f400b0a0 tx=0x7f53f400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 -- 172.21.15.37:0/4018189324 shutdown_connections 2024-04-23T18:19:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 --2- 172.21.15.37:0/4018189324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc102300 0x7f53fc10e7f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 --2- 172.21.15.37:0/4018189324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53fc107990 0x7f53fc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 --2- 172.21.15.37:0/4018189324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53fc101940 0x7f53fc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 -- 172.21.15.37:0/4018189324 >> 172.21.15.37:0/4018189324 conn(0x7f53fc0788f0 msgr2=0x7f53fc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 -- 172.21.15.37:0/4018189324 shutdown_connections 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.503+0000 7f540302b700 1 -- 172.21.15.37:0/4018189324 wait complete. 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 Processor -- start 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 -- start start 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc101940 0x7f53fc199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53fc102300 0x7f53fc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53fc107990 0x7f53fc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc111b00 con 0x7f53fc101940 2024-04-23T18:19:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc111980 con 0x7f53fc107990 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f540302b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc111c80 con 0x7f53fc102300 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f5400dc7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc101940 0x7f53fc199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f5400dc7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc101940 0x7f53fc199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55982/0 (socket says 172.21.15.37:55982) 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f5400dc7700 1 -- 172.21.15.37:0/1100425399 learned_addr learned my addr 172.21.15.37:0/1100425399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f54015c8700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53fc107990 0x7f53fc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.504+0000 7f53fbfff700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53fc102300 0x7f53fc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f5400dc7700 1 -- 172.21.15.37:0/1100425399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53fc102300 msgr2=0x7f53fc199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f5400dc7700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53fc102300 0x7f53fc199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f5400dc7700 1 -- 172.21.15.37:0/1100425399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53fc107990 msgr2=0x7f53fc19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f5400dc7700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53fc107990 0x7f53fc19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f5400dc7700 1 -- 172.21.15.37:0/1100425399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc19e0d0 con 0x7f53fc101940 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f5400dc7700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc101940 0x7f53fc199110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e9c0 tx=0x7f53f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f000cbd0 con 0x7f53fc101940 2024-04-23T18:19:21.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53f000cd70 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0005610 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc19e3c0 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.505+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc1a5c40 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.506+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53fc063190 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.507+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53f0004750 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53f9ffb700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f53ec0a3d50 0x7f53ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53fbfff700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f53ec0a3d50 0x7f53ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f53f00d0b40 con 0x7f53fc101940 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53f9ffb700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] conn(0x7f53ec0ad530 0x7f53ec0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 --> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53f001ee50 con 0x7f53ec0ad530 2024-04-23T18:19:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f53f00d0ef0 con 0x7f53fc101940 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f54015c8700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] conn(0x7f53ec0ad530 0x7f53ec0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.508+0000 7f53fbfff700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f53ec0a3d50 0x7f53ec0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f53fc079040 tx=0x7f53e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.509+0000 7f54015c8700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] conn(0x7f53ec0ad530 0x7f53ec0af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.509+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== osd.6 v2:172.21.15.37:6818/825947067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53f001ee50 con 0x7f53ec0ad530 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.526+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 --> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53fc063610 con 0x7f53ec0ad530 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.526+0000 7f53f9ffb700 1 -- 172.21.15.37:0/1100425399 <== osd.6 v2:172.21.15.37:6818/825947067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f53fc063610 con 0x7f53ec0ad530 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.526+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] conn(0x7f53ec0ad530 msgr2=0x7f53ec0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.526+0000 7f540302b700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] conn(0x7f53ec0ad530 0x7f53ec0af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f53ec0a3d50 msgr2=0x7f53ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f53ec0a3d50 0x7f53ec0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f53fc079040 tx=0x7f53e8009260 comp rx=0 tx=0).stop 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc101940 msgr2=0x7f53fc199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 --2- 172.21.15.37:0/1100425399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53fc101940 0x7f53fc199110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e9c0 tx=0x7f53f000ecd0 comp rx=0 tx=0).stop 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f54015c8700 1 -- 172.21.15.37:0/1100425399 reap_dead start 2024-04-23T18:19:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 shutdown_connections 2024-04-23T18:19:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 >> 172.21.15.37:0/1100425399 conn(0x7f53fc0788f0 msgr2=0x7f53fc06aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 shutdown_connections 2024-04-23T18:19:21.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:21.527+0000 7f540302b700 1 -- 172.21.15.37:0/1100425399 wait complete. 2024-04-23T18:19:21.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:21 smithi172 ceph-mon[32456]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T18:19:21.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:21 smithi173 ceph-mon[32906]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T18:19:21.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:21 smithi037 ceph-mon[29404]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T18:19:22.305 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:23.497 INFO:teuthology.orchestra.run.smithi037.stdout:167503724596 2024-04-23T18:19:23.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T18:19:23.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:23 smithi037 ceph-mon[29404]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T18:19:23.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:23 smithi172 ceph-mon[32456]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T18:19:23.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:23 smithi173 ceph-mon[32906]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T18:19:24.192 INFO:teuthology.orchestra.run.smithi037.stdout:201863462959 2024-04-23T18:19:24.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-23T18:19:24.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.391+0000 7f175113b700 1 -- 172.21.15.37:0/1051353914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 msgr2=0x7f174c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.391+0000 7f175113b700 1 --2- 172.21.15.37:0/1051353914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c101490 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1740009990 tx=0x7f1740009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.391+0000 7f175113b700 1 -- 172.21.15.37:0/1051353914 shutdown_connections 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.391+0000 7f175113b700 1 --2- 172.21.15.37:0/1051353914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c1019d0 0x7f174c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.391+0000 7f175113b700 1 --2- 172.21.15.37:0/1051353914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c108ad0 0x7f174c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.391+0000 7f175113b700 1 --2- 172.21.15.37:0/1051353914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c101490 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.392+0000 7f175113b700 1 -- 172.21.15.37:0/1051353914 >> 172.21.15.37:0/1051353914 conn(0x7f174c077fb0 msgr2=0x7f174c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.392+0000 7f175113b700 1 -- 172.21.15.37:0/1051353914 shutdown_connections 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.392+0000 7f175113b700 1 -- 172.21.15.37:0/1051353914 wait complete. 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.392+0000 7f175113b700 1 Processor -- start 2024-04-23T18:19:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 -- start start 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c1019d0 0x7f174c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c108ad0 0x7f174c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f174c112be0 con 0x7f174c101030 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f174c112a60 con 0x7f174c1019d0 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f175113b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f174c112d60 con 0x7f174c108ad0 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f174ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f174ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55998/0 (socket says 172.21.15.37:55998) 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f174ad9d700 1 -- 172.21.15.37:0/1373835497 learned_addr learned my addr 172.21.15.37:0/1373835497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f174a59c700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c1019d0 0x7f174c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.393+0000 7f174b59e700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c108ad0 0x7f174c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174ad9d700 1 -- 172.21.15.37:0/1373835497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c108ad0 msgr2=0x7f174c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174ad9d700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c108ad0 0x7f174c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174ad9d700 1 -- 172.21.15.37:0/1373835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c1019d0 msgr2=0x7f174c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174ad9d700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c1019d0 0x7f174c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174ad9d700 1 -- 172.21.15.37:0/1373835497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c19e0e0 con 0x7f174c101030 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174b59e700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c108ad0 0x7f174c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f174ad9d700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c1990d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c920 tx=0x7f173c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c004a30 con 0x7f174c101030 2024-04-23T18:19:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173c0054d0 con 0x7f174c101030 2024-04-23T18:19:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c19e3d0 con 0x7f174c101030 2024-04-23T18:19:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c102140 con 0x7f174c101030 2024-04-23T18:19:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.394+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c007c70 con 0x7f174c101030 2024-04-23T18:19:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.396+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c069730 con 0x7f174c101030 2024-04-23T18:19:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.397+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f173c004c60 con 0x7f174c101030 2024-04-23T18:19:24.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.397+0000 7f1733fff700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f17340a3cd0 0x7f17340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.398+0000 7f174a59c700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f17340a3cd0 0x7f17340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.398+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f173c0cfe70 con 0x7f174c101030 2024-04-23T18:19:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.398+0000 7f174a59c700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f17340a3cd0 0x7f17340a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f174000b5c0 tx=0x7f1740005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:24.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.407+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f173c014030 con 0x7f174c101030 2024-04-23T18:19:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.660+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f174c02df80 con 0x7f174c101030 2024-04-23T18:19:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.660+0000 7f1733fff700 1 -- 172.21.15.37:0/1373835497 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f173c09c8a0 con 0x7f174c101030 2024-04-23T18:19:24.655 INFO:teuthology.orchestra.run.smithi037.stdout:322122547228 2024-04-23T18:19:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f17340a3cd0 msgr2=0x7f17340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f17340a3cd0 0x7f17340a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f174000b5c0 tx=0x7f1740005960 comp rx=0 tx=0).stop 2024-04-23T18:19:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 msgr2=0x7f174c1990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:24.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c1990d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c920 tx=0x7f173c00cc30 comp rx=0 tx=0).stop 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 shutdown_connections 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f17340a3cd0 0x7f17340a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f174c101030 0x7f174c1990d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f174c108ad0 0x7f174c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 --2- 172.21.15.37:0/1373835497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c1019d0 0x7f174c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 >> 172.21.15.37:0/1373835497 conn(0x7f174c077fb0 msgr2=0x7f174c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 shutdown_connections 2024-04-23T18:19:24.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:24.664+0000 7f175113b700 1 -- 172.21.15.37:0/1373835497 wait complete. 2024-04-23T18:19:25.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:25 smithi172 ceph-mon[32456]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T18:19:25.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:25 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1373835497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:19:25.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:25 smithi173 ceph-mon[32906]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T18:19:25.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:25 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1373835497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:19:25.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:25 smithi037 ceph-mon[29404]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T18:19:25.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:25 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1373835497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:19:26.425 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:26.425 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:27.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:27 smithi172 ceph-mon[32456]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:19:27.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:27 smithi173 ceph-mon[32906]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:19:27.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:27 smithi037 ceph-mon[29404]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:19:27.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547224 got 322122547228 for osd.10 2024-04-23T18:19:27.919 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:29.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:29 smithi037 ceph-mon[29404]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:29.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:29 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1105132928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:19:29.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:29 smithi172 ceph-mon[32456]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:29.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:29 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1105132928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:19:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:29 smithi173 ceph-mon[32906]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:29.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:29 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1105132928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 -- 172.21.15.37:0/1737593796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728102ac0 msgr2=0x7f4728102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 --2- 172.21.15.37:0/1737593796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728102ac0 0x7f4728102f40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4718009990 tx=0x7f4718009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 -- 172.21.15.37:0/1737593796 shutdown_connections 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 --2- 172.21.15.37:0/1737593796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4728103480 0x7f472810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 --2- 172.21.15.37:0/1737593796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728108a70 0x7f4728108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 --2- 172.21.15.37:0/1737593796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728102ac0 0x7f4728102f40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 -- 172.21.15.37:0/1737593796 >> 172.21.15.37:0/1737593796 conn(0x7f4728079bb0 msgr2=0x7f4728079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 -- 172.21.15.37:0/1737593796 shutdown_connections 2024-04-23T18:19:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 -- 172.21.15.37:0/1737593796 wait complete. 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.131+0000 7f472c870700 1 Processor -- start 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 -- start start 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728102ac0 0x7f4728199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4728103480 0x7f4728199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 0x7f472819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4728112c00 con 0x7f4728108a70 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4728112a80 con 0x7f4728102ac0 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472c870700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4728112d80 con 0x7f4728103480 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 0x7f472819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 0x7f472819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39228/0 (socket says 172.21.15.37:39228) 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 -- 172.21.15.37:0/1105132928 learned_addr learned my addr 172.21.15.37:0/1105132928 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472659c700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728102ac0 0x7f4728199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4725d9b700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4728103480 0x7f4728199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 -- 172.21.15.37:0/1105132928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4728103480 msgr2=0x7f4728199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4728103480 0x7f4728199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 -- 172.21.15.37:0/1105132928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728102ac0 msgr2=0x7f4728199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728102ac0 0x7f4728199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 -- 172.21.15.37:0/1105132928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472819e120 con 0x7f4728108a70 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stdout:107374182462 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f472659c700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728102ac0 0x7f4728199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.132+0000 7f4726d9d700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 0x7f472819d9e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4720007ee0 tx=0x7f472000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.133+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472000ddd0 con 0x7f4728108a70 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.133+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472819e410 con 0x7f4728108a70 2024-04-23T18:19:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.133+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47281a5c90 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.133+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4720004020 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.133+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720015660 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.134+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4728069730 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.135+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f47200044f0 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.135+0000 7f47177fe700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f47100a3d50 0x7f47100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.136+0000 7f472659c700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f47100a3d50 0x7f47100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.136+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f47200cff50 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.136+0000 7f472659c700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f47100a3d50 0x7f47100a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f471c00bb90 tx=0x7f471c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.140+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4720010070 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.349+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f472802df80 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.350+0000 7f47177fe700 1 -- 172.21.15.37:0/1105132928 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f472009c880 con 0x7f4728108a70 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f47100a3d50 msgr2=0x7f47100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f47100a3d50 0x7f47100a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f471c00bb90 tx=0x7f471c005d80 comp rx=0 tx=0).stop 2024-04-23T18:19:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 msgr2=0x7f472819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 0x7f472819d9e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4720007ee0 tx=0x7f472000d3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 shutdown_connections 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f47100a3d50 0x7f47100a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4728108a70 0x7f472819d9e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4728102ac0 0x7f4728199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 --2- 172.21.15.37:0/1105132928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4728103480 0x7f4728199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.352+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 >> 172.21.15.37:0/1105132928 conn(0x7f4728079bb0 msgr2=0x7f47281018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.353+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 shutdown_connections 2024-04-23T18:19:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:29.353+0000 7f472c870700 1 -- 172.21.15.37:0/1105132928 wait complete. 2024-04-23T18:19:30.804 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:30 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2920231283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:19:30.830 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182457 got 107374182462 for osd.3 2024-04-23T18:19:30.830 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:30.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:30 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2920231283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:19:30.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:30 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2920231283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 -- 172.21.15.37:0/2687430375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb0108a60 msgr2=0x7fbdb0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 --2- 172.21.15.37:0/2687430375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb0108a60 0x7fbdb0108e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009990 tx=0x7fbda0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 -- 172.21.15.37:0/2687430375 shutdown_connections 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 --2- 172.21.15.37:0/2687430375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb0108a60 0x7fbdb0108e40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 --2- 172.21.15.37:0/2687430375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdb006bbb0 0x7fbdb010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 --2- 172.21.15.37:0/2687430375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdb006b1f0 0x7fbdb006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 -- 172.21.15.37:0/2687430375 >> 172.21.15.37:0/2687430375 conn(0x7fbdb00789e0 msgr2=0x7fbdb0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 -- 172.21.15.37:0/2687430375 shutdown_connections 2024-04-23T18:19:31.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.111+0000 7fbdb494a700 1 -- 172.21.15.37:0/2687430375 wait complete. 2024-04-23T18:19:31.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 Processor -- start 2024-04-23T18:19:31.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 -- start start 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdb006b1f0 0x7fbdb0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 0x7fbdb0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdb0108a60 0x7fbdb019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0112cb0 con 0x7fbdb006bbb0 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0112b30 con 0x7fbdb0108a60 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.112+0000 7fbdb494a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0112e30 con 0x7fbdb006b1f0 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 0x7fbdb0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdae59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdb006b1f0 0x7fbdb0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 0x7fbdb0199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39262/0 (socket says 172.21.15.37:39262) 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 -- 172.21.15.37:0/1766392817 learned_addr learned my addr 172.21.15.37:0/1766392817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdaed9d700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdb0108a60 0x7fbdb019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 -- 172.21.15.37:0/1766392817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdb006b1f0 msgr2=0x7fbdb0199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdb006b1f0 0x7fbdb0199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 -- 172.21.15.37:0/1766392817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdb0108a60 msgr2=0x7fbdb019d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdb0108a60 0x7fbdb019d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 -- 172.21.15.37:0/1766392817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb019e130 con 0x7fbdb006bbb0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdadd9b700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 0x7fbdb0199660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbda400d980 tx=0x7fbda400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4016070 con 0x7fbdb006bbb0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda4004830 con 0x7fbdb006bbb0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb019e420 con 0x7fbdb006bbb0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4005540 con 0x7fbdb006bbb0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.113+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01a5ca0 con 0x7fbdb006bbb0 2024-04-23T18:19:31.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.114+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb010b060 con 0x7fbdb006bbb0 2024-04-23T18:19:31.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.116+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbda4004220 con 0x7fbdb006bbb0 2024-04-23T18:19:31.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.116+0000 7fbd9f7fe700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd980a3cd0 0x7fbd980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.116+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fbda40d0680 con 0x7fbdb006bbb0 2024-04-23T18:19:31.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.116+0000 7fbdae59c700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd980a3cd0 0x7fbd980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.117+0000 7fbdae59c700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd980a3cd0 0x7fbd980a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbda0006010 tx=0x7fbda000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:31.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.120+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbda401f020 con 0x7fbdb006bbb0 2024-04-23T18:19:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 -- 172.21.15.37:0/3277117307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 msgr2=0x7f6698108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 --2- 172.21.15.37:0/3277117307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698108e40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f668800b0a0 tx=0x7f668800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 -- 172.21.15.37:0/3277117307 shutdown_connections 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 --2- 172.21.15.37:0/3277117307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698108e40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 --2- 172.21.15.37:0/3277117307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f669806bbb0 0x7f669810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 --2- 172.21.15.37:0/3277117307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f669806b1f0 0x7f669806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 -- 172.21.15.37:0/3277117307 >> 172.21.15.37:0/3277117307 conn(0x7f66980789e0 msgr2=0x7f6698078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 -- 172.21.15.37:0/3277117307 shutdown_connections 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.265+0000 7f669fd1e700 1 -- 172.21.15.37:0/3277117307 wait complete. 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 Processor -- start 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 -- start start 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f669806b1f0 0x7f66981019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f669806bbb0 0x7f6698101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112cf0 con 0x7f6698108a60 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112b70 con 0x7f669806bbb0 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669fd1e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112e70 con 0x7f669806b1f0 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669e2bb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669e2bb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39242/0 (socket says 172.21.15.37:39242) 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669e2bb700 1 -- 172.21.15.37:0/2920231283 learned_addr learned my addr 172.21.15.37:0/2920231283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669daba700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f669806b1f0 0x7f66981019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.266+0000 7f669d2b9700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f669806bbb0 0x7f6698101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.182 INFO:teuthology.orchestra.run.smithi037.stdout:34359738446 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669e2bb700 1 -- 172.21.15.37:0/2920231283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f669806b1f0 msgr2=0x7f66981019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669e2bb700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f669806b1f0 0x7f66981019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669e2bb700 1 -- 172.21.15.37:0/2920231283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f669806bbb0 msgr2=0x7f6698101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669e2bb700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f669806bbb0 0x7f6698101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669e2bb700 1 -- 172.21.15.37:0/2920231283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66981a7700 con 0x7f6698108a60 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669e2bb700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698102430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f669000c970 tx=0x7f669000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690007b00 con 0x7f6698108a60 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66981a79f0 con 0x7f6698108a60 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6690004ce0 con 0x7f6698108a60 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690005530 con 0x7f6698108a60 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.267+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a7ee0 con 0x7f6698108a60 2024-04-23T18:19:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.268+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669810b060 con 0x7f6698108a60 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.269+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6690004750 con 0x7f6698108a60 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.270+0000 7f6696ffd700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66800a3cd0 0x7f66800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.270+0000 7f669daba700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66800a3cd0 0x7f66800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.270+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f66900d0510 con 0x7f6698108a60 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.270+0000 7f669daba700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66800a3cd0 0x7f66800a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6688009e60 tx=0x7f668800b7d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.273+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6690014030 con 0x7f6698108a60 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.482+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f669810aa40 con 0x7f6698108a60 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.483+0000 7f6696ffd700 1 -- 172.21.15.37:0/2920231283 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f669009ce40 con 0x7f6698108a60 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66800a3cd0 msgr2=0x7f66800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66800a3cd0 0x7f66800a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6688009e60 tx=0x7f668800b7d0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 msgr2=0x7f6698102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698102430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f669000c970 tx=0x7f669000cd30 comp rx=0 tx=0).stop 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 shutdown_connections 2024-04-23T18:19:31.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66800a3cd0 0x7f66800a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698108a60 0x7f6698102430 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f669806b1f0 0x7f66981019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 --2- 172.21.15.37:0/2920231283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f669806bbb0 0x7f6698101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 >> 172.21.15.37:0/2920231283 conn(0x7f66980789e0 msgr2=0x7f66980ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:31.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 shutdown_connections 2024-04-23T18:19:31.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:30.485+0000 7f669fd1e700 1 -- 172.21.15.37:0/2920231283 wait complete. 2024-04-23T18:19:31.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.329+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fbdb019a480 con 0x7fbdb006bbb0 2024-04-23T18:19:31.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.330+0000 7fbd9f7fe700 1 -- 172.21.15.37:0/1766392817 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbda409d0b0 con 0x7fbdb006bbb0 2024-04-23T18:19:31.323 INFO:teuthology.orchestra.run.smithi037.stdout:296352743458 2024-04-23T18:19:31.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd980a3cd0 msgr2=0x7fbd980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd980a3cd0 0x7fbd980a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbda0006010 tx=0x7fbda000b540 comp rx=0 tx=0).stop 2024-04-23T18:19:31.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 msgr2=0x7fbdb0199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 0x7fbdb0199660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbda400d980 tx=0x7fbda400dc90 comp rx=0 tx=0).stop 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 shutdown_connections 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbd980a3cd0 0x7fbd980a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdb0108a60 0x7fbdb019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdb006b1f0 0x7fbdb0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 --2- 172.21.15.37:0/1766392817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdb006bbb0 0x7fbdb0199660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 >> 172.21.15.37:0/1766392817 conn(0x7fbdb00789e0 msgr2=0x7fbdb00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 shutdown_connections 2024-04-23T18:19:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:31.332+0000 7fbdb494a700 1 -- 172.21.15.37:0/1766392817 wait complete. 2024-04-23T18:19:31.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:31 smithi037 ceph-mon[29404]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:19:31.621 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:31 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1766392817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:19:31.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:31 smithi172 ceph-mon[32456]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:19:31.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:31 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1766392817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:19:31.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:31 smithi173 ceph-mon[32906]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:19:31.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:31 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1766392817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:19:31.983 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738446 for osd.0 2024-04-23T18:19:31.983 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:32.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743455 got 296352743458 for osd.9 2024-04-23T18:19:32.598 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:32.617 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:33.061 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:33.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:33 smithi037 ceph-mon[29404]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T18:19:33.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:33 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:33 smithi172 ceph-mon[32456]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T18:19:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:33 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:33 smithi173 ceph-mon[32906]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T18:19:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:33 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:35.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.052+0000 7f0da6bbe700 1 -- 172.21.15.37:0/515619959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 msgr2=0x7f0da010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.052+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/515619959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da010f8b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800b0a0 tx=0x7f0d9800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 -- 172.21.15.37:0/515619959 shutdown_connections 2024-04-23T18:19:35.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/515619959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da010f8b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/515619959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0108a50 0x7f0da0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/515619959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 0x7f0da0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 -- 172.21.15.37:0/515619959 >> 172.21.15.37:0/515619959 conn(0x7f0da0079ba0 msgr2=0x7f0da0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:35.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 -- 172.21.15.37:0/515619959 shutdown_connections 2024-04-23T18:19:35.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 -- 172.21.15.37:0/515619959 wait complete. 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 Processor -- start 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 -- start start 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.053+0000 7f0da6bbe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 0x7f0da0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da6bbe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da6bbe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0108a50 0x7f0da00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da6bbe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0112c20 con 0x7f0da0103450 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da6bbe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0112aa0 con 0x7f0da0108a50 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da6bbe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0112da0 con 0x7f0da0102a90 2024-04-23T18:19:35.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da495a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 0x7f0da0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da0075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37928/0 (socket says 172.21.15.37:37928) 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 -- 172.21.15.37:0/3565429577 learned_addr learned my addr 172.21.15.37:0/3565429577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da515b700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0108a50 0x7f0da00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 -- 172.21.15.37:0/3565429577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 msgr2=0x7f0da0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 0x7f0da0074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 -- 172.21.15.37:0/3565429577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0108a50 msgr2=0x7f0da00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0108a50 0x7f0da00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 -- 172.21.15.37:0/3565429577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da01a7630 con 0x7f0da0103450 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da495a700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 0x7f0da0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9ffff700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da0075280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e9c0 tx=0x7f0d8c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:35.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00cbd0 con 0x7f0da0103450 2024-04-23T18:19:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c00ee60 con 0x7f0da0103450 2024-04-23T18:19:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da01a7920 con 0x7f0da0103450 2024-04-23T18:19:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.054+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0055c0 con 0x7f0da0103450 2024-04-23T18:19:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.055+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da01a7e10 con 0x7f0da0103450 2024-04-23T18:19:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.056+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da0069730 con 0x7f0da0103450 2024-04-23T18:19:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.057+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0d8c004750 con 0x7f0da0103450 2024-04-23T18:19:35.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.057+0000 7f0d9dffb700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d900a3cd0 0x7f0d900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.057+0000 7f0da495a700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d900a3cd0 0x7f0d900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.057+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0d8c0d0670 con 0x7f0da0103450 2024-04-23T18:19:35.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.058+0000 7f0da495a700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d900a3cd0 0x7f0d900a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009fd0 tx=0x7f0d94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.062+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d8c09d0a0 con 0x7f0da0103450 2024-04-23T18:19:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.274+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0da002df80 con 0x7f0da0103450 2024-04-23T18:19:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.275+0000 7f0d9dffb700 1 -- 172.21.15.37:0/3565429577 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d8c09c9a0 con 0x7f0da0103450 2024-04-23T18:19:35.268 INFO:teuthology.orchestra.run.smithi037.stdout:77309411397 2024-04-23T18:19:35.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d900a3cd0 msgr2=0x7f0d900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d900a3cd0 0x7f0d900a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009fd0 tx=0x7f0d94009260 comp rx=0 tx=0).stop 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 msgr2=0x7f0da0075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da0075280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e9c0 tx=0x7f0d8c00ecd0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 shutdown_connections 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d900a3cd0 0x7f0d900a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0108a50 0x7f0da00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0da0102a90 0x7f0da0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 --2- 172.21.15.37:0/3565429577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da0103450 0x7f0da0075280 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 >> 172.21.15.37:0/3565429577 conn(0x7f0da0079ba0 msgr2=0x7f0da0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 shutdown_connections 2024-04-23T18:19:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.277+0000 7f0da6bbe700 1 -- 172.21.15.37:0/3565429577 wait complete. 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 -- 172.21.15.37:0/428208941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67741026e0 msgr2=0x7f677410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 --2- 172.21.15.37:0/428208941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67741026e0 0x7f677410e770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f676c00b0a0 tx=0x7f676c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 -- 172.21.15.37:0/428208941 shutdown_connections 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 --2- 172.21.15.37:0/428208941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67741026e0 0x7f677410e770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 --2- 172.21.15.37:0/428208941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774106990 0x7f6774106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 --2- 172.21.15.37:0/428208941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6774101d20 0x7f67741021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 -- 172.21.15.37:0/428208941 >> 172.21.15.37:0/428208941 conn(0x7f67740788f0 msgr2=0x7f6774078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:35.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 -- 172.21.15.37:0/428208941 shutdown_connections 2024-04-23T18:19:35.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.279+0000 7f677c0a7700 1 -- 172.21.15.37:0/428208941 wait complete. 2024-04-23T18:19:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 Processor -- start 2024-04-23T18:19:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 -- start start 2024-04-23T18:19:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 0x7f6774199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67741026e0 0x7f677419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774106990 0x7f67741a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6774111ae0 con 0x7f6774101d20 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6774111960 con 0x7f6774106990 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677c0a7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6774111c60 con 0x7f67741026e0 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 0x7f6774199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 0x7f6774199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37938/0 (socket says 172.21.15.37:37938) 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 -- 172.21.15.37:0/638805619 learned_addr learned my addr 172.21.15.37:0/638805619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779642700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67741026e0 0x7f677419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f677a644700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774106990 0x7f67741a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 -- 172.21.15.37:0/638805619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67741026e0 msgr2=0x7f677419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67741026e0 0x7f677419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 -- 172.21.15.37:0/638805619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774106990 msgr2=0x7f67741a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774106990 0x7f67741a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.280+0000 7f6779e43700 1 -- 172.21.15.37:0/638805619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67741a53c0 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.281+0000 7f6779e43700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 0x7f6774199e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f677000c9b0 tx=0x7f677000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.281+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770007b00 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.281+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677000ce50 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.281+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67741a55f0 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.281+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770018660 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.281+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a5b00 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.282+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774103080 con 0x7f6774101d20 2024-04-23T18:19:35.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.283+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f677000f450 con 0x7f6774101d20 2024-04-23T18:19:35.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.284+0000 7f676affd700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f67600a3cd0 0x7f67600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:35.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.285+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f67700d04f0 con 0x7f6774101d20 2024-04-23T18:19:35.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.285+0000 7f6779642700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f67600a3cd0 0x7f67600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:35.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.286+0000 7f6779642700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f67600a3cd0 0x7f67600a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6774078e80 tx=0x7f676400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:35.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.289+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6770014030 con 0x7f6774101d20 2024-04-23T18:19:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.504+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f677402df80 con 0x7f6774101d20 2024-04-23T18:19:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.505+0000 7f676affd700 1 -- 172.21.15.37:0/638805619 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f677009cfb0 con 0x7f6774101d20 2024-04-23T18:19:35.498 INFO:teuthology.orchestra.run.smithi037.stdout:261993005096 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f67600a3cd0 msgr2=0x7f67600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f67600a3cd0 0x7f67600a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6774078e80 tx=0x7f676400b410 comp rx=0 tx=0).stop 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 msgr2=0x7f6774199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 0x7f6774199e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f677000c9b0 tx=0x7f677000ccc0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 shutdown_connections 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f67600a3cd0 0x7f67600a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6774101d20 0x7f6774199e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774106990 0x7f67741a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 --2- 172.21.15.37:0/638805619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67741026e0 0x7f677419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:35.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 >> 172.21.15.37:0/638805619 conn(0x7f67740788f0 msgr2=0x7f67740ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:35.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 shutdown_connections 2024-04-23T18:19:35.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:35.507+0000 7f677c0a7700 1 -- 172.21.15.37:0/638805619 wait complete. 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3565429577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:19:35.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/638805619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:19:36.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:19:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3565429577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:19:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/638805619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3565429577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:19:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/638805619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:19:37.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411391 got 77309411397 for osd.2 2024-04-23T18:19:37.539 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:37.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005091 got 261993005096 for osd.8 2024-04-23T18:19:37.647 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:37.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:37 smithi037 ceph-mon[29404]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-23T18:19:38.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:37 smithi172 ceph-mon[32456]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-23T18:19:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:37 smithi173 ceph-mon[32906]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-23T18:19:39.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 -- 172.21.15.37:0/2862084755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8107990 msgr2=0x7f23a8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:39.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 --2- 172.21.15.37:0/2862084755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8107990 0x7f23a8107d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2398009990 tx=0x7f2398009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 -- 172.21.15.37:0/2862084755 shutdown_connections 2024-04-23T18:19:39.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 --2- 172.21.15.37:0/2862084755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8107990 0x7f23a8107d70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 --2- 172.21.15.37:0/2862084755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a8102300 0x7f23a810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 --2- 172.21.15.37:0/2862084755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8101940 0x7f23a8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 -- 172.21.15.37:0/2862084755 >> 172.21.15.37:0/2862084755 conn(0x7f23a80788f0 msgr2=0x7f23a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 -- 172.21.15.37:0/2862084755 shutdown_connections 2024-04-23T18:19:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.455+0000 7f23ae530700 1 -- 172.21.15.37:0/2862084755 wait complete. 2024-04-23T18:19:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 Processor -- start 2024-04-23T18:19:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 -- start start 2024-04-23T18:19:39.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a8101940 0x7f23a8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 0x7f23a819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8107990 0x7f23a81a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8111b00 con 0x7f23a8102300 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8111980 con 0x7f23a8107990 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f23ae530700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8111c80 con 0x7f23a8101940 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f239ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 0x7f23a819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f239ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 0x7f23a819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37966/0 (socket says 172.21.15.37:37966) 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.456+0000 7f239ffff700 1 -- 172.21.15.37:0/2543590809 learned_addr learned my addr 172.21.15.37:0/2543590809 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23a7fff700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a8101940 0x7f23a8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23acacd700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8107990 0x7f23a81a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f239ffff700 1 -- 172.21.15.37:0/2543590809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a8101940 msgr2=0x7f23a8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f239ffff700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a8101940 0x7f23a8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f239ffff700 1 -- 172.21.15.37:0/2543590809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8107990 msgr2=0x7f23a81a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f239ffff700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8107990 0x7f23a81a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f239ffff700 1 -- 172.21.15.37:0/2543590809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a81a5500 con 0x7f23a8102300 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23acacd700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8107990 0x7f23a81a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f239ffff700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 0x7f23a819a480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f239000c9b0 tx=0x7f239000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390007b00 con 0x7f23a8102300 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2390007ca0 con 0x7f23a8102300 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a81a5790 con 0x7f23a8102300 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390018730 con 0x7f23a8102300 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.457+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a81a5cf0 con 0x7f23a8102300 2024-04-23T18:19:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.458+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a8109eb0 con 0x7f23a8102300 2024-04-23T18:19:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.459+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f239000f450 con 0x7f23a8102300 2024-04-23T18:19:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.460+0000 7f23a5ffb700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23940a3d50 0x7f23940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.460+0000 7f23a7fff700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23940a3d50 0x7f23940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.460+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f23900d04a0 con 0x7f23a8102300 2024-04-23T18:19:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.460+0000 7f23a7fff700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23940a3d50 0x7f23940a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2398006010 tx=0x7f239801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.465+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2390014030 con 0x7f23a8102300 2024-04-23T18:19:39.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.674+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f23a8109890 con 0x7f23a8102300 2024-04-23T18:19:39.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.675+0000 7f23a5ffb700 1 -- 172.21.15.37:0/2543590809 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f239009cdd0 con 0x7f23a8102300 2024-04-23T18:19:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:227633266733 2024-04-23T18:19:39.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23940a3d50 msgr2=0x7f23940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:39.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23940a3d50 0x7f23940a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2398006010 tx=0x7f239801a040 comp rx=0 tx=0).stop 2024-04-23T18:19:39.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 msgr2=0x7f23a819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:39.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 0x7f23a819a480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f239000c9b0 tx=0x7f239000cd70 comp rx=0 tx=0).stop 2024-04-23T18:19:39.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 shutdown_connections 2024-04-23T18:19:39.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f23940a3d50 0x7f23940a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a8107990 0x7f23a81a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a8101940 0x7f23a8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 --2- 172.21.15.37:0/2543590809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a8102300 0x7f23a819a480 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:39.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 >> 172.21.15.37:0/2543590809 conn(0x7f23a80788f0 msgr2=0x7f23a81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:39.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 shutdown_connections 2024-04-23T18:19:39.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:39.678+0000 7f23ae530700 1 -- 172.21.15.37:0/2543590809 wait complete. 2024-04-23T18:19:39.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:39 smithi037 ceph-mon[29404]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:40.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:39 smithi172 ceph-mon[32456]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:40.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:39 smithi173 ceph-mon[32906]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2543590809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9, 5, 8]}]: dispatch 2024-04-23T18:19:40.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-04-23T18:19:40.731 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:19:40.731 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:40 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-23T18:19:41.044 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266727 got 227633266733 for osd.7 2024-04-23T18:19:41.044 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:41.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2543590809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:19:41.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9, 5, 8]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:19:41.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:40 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-23T18:19:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2543590809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:19:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-23T18:19:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-23T18:19:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-23T18:19:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]: dispatch 2024-04-23T18:19:41.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-23T18:19:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9, 5, 8]}]: dispatch 2024-04-23T18:19:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-04-23T18:19:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:19:41.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:40 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 -- 172.21.15.37:0/1209188912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020102ac0 msgr2=0x7f1020102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 --2- 172.21.15.37:0/1209188912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020102ac0 0x7f1020102f40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f101c009990 tx=0x7f101c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 -- 172.21.15.37:0/1209188912 shutdown_connections 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 --2- 172.21.15.37:0/1209188912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020103480 0x7f102010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 --2- 172.21.15.37:0/1209188912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1020108a70 0x7f1020108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 --2- 172.21.15.37:0/1209188912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020102ac0 0x7f1020102f40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 -- 172.21.15.37:0/1209188912 >> 172.21.15.37:0/1209188912 conn(0x7f1020079bb0 msgr2=0x7f1020079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 -- 172.21.15.37:0/1209188912 shutdown_connections 2024-04-23T18:19:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.297+0000 7f1027385700 1 -- 172.21.15.37:0/1209188912 wait complete. 2024-04-23T18:19:41.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 Processor -- start 2024-04-23T18:19:41.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 -- start start 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1020102ac0 0x7f1020074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 0x7f1020075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020108a70 0x7f10200757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1020112c00 con 0x7f1020103480 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1020112a80 con 0x7f1020108a70 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1027385700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1020112d80 con 0x7f1020102ac0 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.298+0000 7f1024920700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 0x7f1020075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 0x7f1020075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37982/0 (socket says 172.21.15.37:37982) 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 -- 172.21.15.37:0/2008350419 learned_addr learned my addr 172.21.15.37:0/2008350419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1025121700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1020102ac0 0x7f1020074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1025922700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020108a70 0x7f10200757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:41.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 -- 172.21.15.37:0/2008350419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1020102ac0 msgr2=0x7f1020074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1020102ac0 0x7f1020074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 -- 172.21.15.37:0/2008350419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020108a70 msgr2=0x7f10200757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020108a70 0x7f10200757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 -- 172.21.15.37:0/2008350419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10201a7620 con 0x7f1020103480 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1025922700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020108a70 0x7f10200757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1024920700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 0x7f1020075280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f101c0057c0 tx=0x7f101c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c027070 con 0x7f1020103480 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f101c004020 con 0x7f1020103480 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10201a78b0 con 0x7f1020103480 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c022c40 con 0x7f1020103480 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.299+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10201a7de0 con 0x7f1020103480 2024-04-23T18:19:41.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.300+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1020069730 con 0x7f1020103480 2024-04-23T18:19:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.301+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f101c02a070 con 0x7f1020103480 2024-04-23T18:19:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.302+0000 7f10127fc700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f100c0a3d50 0x7f100c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.302+0000 7f1025121700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f100c0a3d50 0x7f100c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.303+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f101c0d1cf0 con 0x7f1020103480 2024-04-23T18:19:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.303+0000 7f1025121700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f100c0a3d50 0x7f100c0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f101400dad0 tx=0x7f101400d4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:41.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.306+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f101c031020 con 0x7f1020103480 2024-04-23T18:19:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.521+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f102002e140 con 0x7f1020103480 2024-04-23T18:19:41.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.521+0000 7f10127fc700 1 -- 172.21.15.37:0/2008350419 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f101c013070 con 0x7f1020103480 2024-04-23T18:19:41.514 INFO:teuthology.orchestra.run.smithi037.stdout:141733920829 2024-04-23T18:19:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f100c0a3d50 msgr2=0x7f100c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f100c0a3d50 0x7f100c0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f101400dad0 tx=0x7f101400d4b0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 msgr2=0x7f1020075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 0x7f1020075280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f101c0057c0 tx=0x7f101c0057f0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 shutdown_connections 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f100c0a3d50 0x7f100c0a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1020108a70 0x7f10200757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1020102ac0 0x7f1020074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 --2- 172.21.15.37:0/2008350419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1020103480 0x7f1020075280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 >> 172.21.15.37:0/2008350419 conn(0x7f1020079bb0 msgr2=0x7f1020101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 shutdown_connections 2024-04-23T18:19:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:41.523+0000 7f1027385700 1 -- 172.21.15.37:0/2008350419 wait complete. 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9, 5, 8]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-04-23T18:19:41.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:19:41.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-23T18:19:41.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:19:41.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:41 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2008350419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:19:42.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9, 5, 8]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:19:42.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:41 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2008350419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:19:42.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:19:42.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-23T18:19:42.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-23T18:19:42.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-23T18:19:42.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 6]}]': finished 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9, 5, 8]}]': finished 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:19:42.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:41 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2008350419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:19:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 -- 172.21.15.37:0/1382338371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 msgr2=0x7f593c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 --2- 172.21.15.37:0/1382338371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c101dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5930009990 tx=0x7f5930009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 -- 172.21.15.37:0/1382338371 shutdown_connections 2024-04-23T18:19:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 --2- 172.21.15.37:0/1382338371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f593c102300 0x7f593c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 --2- 172.21.15.37:0/1382338371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 0x7f593c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 --2- 172.21.15.37:0/1382338371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c101dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 -- 172.21.15.37:0/1382338371 >> 172.21.15.37:0/1382338371 conn(0x7f593c0788f0 msgr2=0x7f593c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 -- 172.21.15.37:0/1382338371 shutdown_connections 2024-04-23T18:19:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.336+0000 7f594093f700 1 -- 172.21.15.37:0/1382338371 wait complete. 2024-04-23T18:19:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 Processor -- start 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 -- start start 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f593c102300 0x7f593c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 0x7f593c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f593c111b50 con 0x7f593c101940 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1119d0 con 0x7f593c107990 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f594093f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f593c111cd0 con 0x7f593c102300 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c1034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38002/0 (socket says 172.21.15.37:38002) 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 -- 172.21.15.37:0/2961687443 learned_addr learned my addr 172.21.15.37:0/2961687443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f5939d9b700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f593c102300 0x7f593c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593ad9d700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 0x7f593c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 -- 172.21.15.37:0/2961687443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f593c102300 msgr2=0x7f593c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f593c102300 0x7f593c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 -- 172.21.15.37:0/2961687443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 msgr2=0x7f593c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 0x7f593c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 -- 172.21.15.37:0/2961687443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c1a76f0 con 0x7f593c101940 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593ad9d700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 0x7f593c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.337+0000 7f593a59c700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c1034f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f592c00d950 tx=0x7f592c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.338+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c016070 con 0x7f593c101940 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.338+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c1a79e0 con 0x7f593c101940 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.338+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f592c009df0 con 0x7f593c101940 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.338+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c00f3c0 con 0x7f593c101940 2024-04-23T18:19:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.338+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a7f40 con 0x7f593c101940 2024-04-23T18:19:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.339+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c069730 con 0x7f593c101940 2024-04-23T18:19:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.340+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f592c019070 con 0x7f593c101940 2024-04-23T18:19:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.341+0000 7f592b7fe700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59240a3cd0 0x7f59240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.341+0000 7f5939d9b700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59240a3cd0 0x7f59240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.341+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f592c0d0160 con 0x7f593c101940 2024-04-23T18:19:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.341+0000 7f5939d9b700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59240a3cd0 0x7f59240a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f59300098e0 tx=0x7f5930005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.344+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f592c09c9d0 con 0x7f593c101940 2024-04-23T18:19:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.554+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f593c02df80 con 0x7f593c101940 2024-04-23T18:19:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.555+0000 7f592b7fe700 1 -- 172.21.15.37:0/2961687443 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f592c09c2d0 con 0x7f593c101940 2024-04-23T18:19:42.548 INFO:teuthology.orchestra.run.smithi037.stdout:55834574923 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59240a3cd0 msgr2=0x7f59240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59240a3cd0 0x7f59240a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f59300098e0 tx=0x7f5930005960 comp rx=0 tx=0).stop 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 msgr2=0x7f593c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c1034f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f592c00d950 tx=0x7f592c00dc60 comp rx=0 tx=0).stop 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 shutdown_connections 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59240a3cd0 0x7f59240a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593c101940 0x7f593c1034f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f593c107990 0x7f593c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 --2- 172.21.15.37:0/2961687443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f593c102300 0x7f593c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 >> 172.21.15.37:0/2961687443 conn(0x7f593c0788f0 msgr2=0x7f593c06ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:42.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 shutdown_connections 2024-04-23T18:19:42.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:42.557+0000 7f594093f700 1 -- 172.21.15.37:0/2961687443 wait complete. 2024-04-23T18:19:42.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:42 smithi037 ceph-mon[29404]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:19:42.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:42 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2961687443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:19:43.077 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920829 for osd.4 2024-04-23T18:19:43.077 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:43.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:42 smithi172 ceph-mon[32456]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:19:43.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:42 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2961687443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:19:43.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:42 smithi173 ceph-mon[32906]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:19:43.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:42 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2961687443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:19:43.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574923 for osd.1 2024-04-23T18:19:43.509 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:43.750 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:43 smithi037 ceph-mon[29404]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:19:43.750 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:43 smithi037 ceph-mon[29404]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-23T18:19:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 -- 172.21.15.37:0/675764143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 msgr2=0x7f670010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 --2- 172.21.15.37:0/675764143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670010ef60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009990 tx=0x7f66f4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 -- 172.21.15.37:0/675764143 shutdown_connections 2024-04-23T18:19:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 --2- 172.21.15.37:0/675764143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670010f4a0 0x7f6700111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 --2- 172.21.15.37:0/675764143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6700074c70 0x7f6700075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 --2- 172.21.15.37:0/675764143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670010ef60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 -- 172.21.15.37:0/675764143 >> 172.21.15.37:0/675764143 conn(0x7f6700079ad0 msgr2=0x7f6700079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 -- 172.21.15.37:0/675764143 shutdown_connections 2024-04-23T18:19:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.965+0000 7f6706b71700 1 -- 172.21.15.37:0/675764143 wait complete. 2024-04-23T18:19:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 Processor -- start 2024-04-23T18:19:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 -- start start 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6700074c70 0x7f670019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670010f4a0 0x7f670019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6700114e70 con 0x7f6700075590 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6700114cf0 con 0x7f6700074c70 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f6706b71700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6700114ff0 con 0x7f670010f4a0 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670019b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38006/0 (socket says 172.21.15.37:38006) 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 -- 172.21.15.37:0/1524988815 learned_addr learned my addr 172.21.15.37:0/1524988815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f670510e700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670010f4a0 0x7f670019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f670490d700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6700074c70 0x7f670019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 -- 172.21.15.37:0/1524988815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670010f4a0 msgr2=0x7f670019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670010f4a0 0x7f670019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 -- 172.21.15.37:0/1524988815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6700074c70 msgr2=0x7f670019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6700074c70 0x7f670019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 -- 172.21.15.37:0/1524988815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67001a02d0 con 0x7f6700075590 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.966+0000 7f66fffff700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670019b800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66f4000c00 tx=0x7f66f40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.967+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4027070 con 0x7f6700075590 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.967+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67001a0560 con 0x7f6700075590 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.967+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f40041d0 con 0x7f6700075590 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.967+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f400f460 con 0x7f6700075590 2024-04-23T18:19:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.967+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67001a7e40 con 0x7f6700075590 2024-04-23T18:19:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.968+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6700069730 con 0x7f6700075590 2024-04-23T18:19:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.969+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f66f402a070 con 0x7f6700075590 2024-04-23T18:19:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.970+0000 7f66fdffb700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66e80a3cd0 0x7f66e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.970+0000 7f670490d700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66e80a3cd0 0x7f66e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.970+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f66f40d15e0 con 0x7f6700075590 2024-04-23T18:19:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.970+0000 7f670490d700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66e80a3cd0 0x7f66e80a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f66f800e270 tx=0x7f66f8009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:43.974+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66f405eff0 con 0x7f6700075590 2024-04-23T18:19:44.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:43 smithi172 ceph-mon[32456]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:19:44.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:43 smithi172 ceph-mon[32456]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-23T18:19:44.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:43 smithi173 ceph-mon[32906]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:19:44.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:43 smithi173 ceph-mon[32906]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-23T18:19:44.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.188+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f670002df80 con 0x7f6700075590 2024-04-23T18:19:44.182 INFO:teuthology.orchestra.run.smithi037.stdout:167503724601 2024-04-23T18:19:44.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.188+0000 7f66fdffb700 1 -- 172.21.15.37:0/1524988815 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66f4013070 con 0x7f6700075590 2024-04-23T18:19:44.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66e80a3cd0 msgr2=0x7f66e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:44.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66e80a3cd0 0x7f66e80a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f66f800e270 tx=0x7f66f8009040 comp rx=0 tx=0).stop 2024-04-23T18:19:44.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 msgr2=0x7f670019b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:44.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670019b800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f66f4000c00 tx=0x7f66f40047b0 comp rx=0 tx=0).stop 2024-04-23T18:19:44.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 shutdown_connections 2024-04-23T18:19:44.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f66e80a3cd0 0x7f66e80a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670010f4a0 0x7f670019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6700074c70 0x7f670019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 --2- 172.21.15.37:0/1524988815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6700075590 0x7f670019b800 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 >> 172.21.15.37:0/1524988815 conn(0x7f6700079ad0 msgr2=0x7f67000ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 shutdown_connections 2024-04-23T18:19:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:44.191+0000 7f6706b71700 1 -- 172.21.15.37:0/1524988815 wait complete. 2024-04-23T18:19:44.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:44 smithi037 ceph-mon[29404]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:19:44.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:44 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1524988815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:19:44.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724601 for osd.5 2024-04-23T18:19:44.905 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:45.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:44 smithi172 ceph-mon[32456]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:19:45.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:44 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1524988815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:19:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:44 smithi173 ceph-mon[32906]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:19:45.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:44 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1524988815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:19:45.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 -- 172.21.15.37:0/3812901059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b8108a60 msgr2=0x7f59b8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 --2- 172.21.15.37:0/3812901059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b8108a60 0x7f59b8108e40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009990 tx=0x7f59b4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 -- 172.21.15.37:0/3812901059 shutdown_connections 2024-04-23T18:19:45.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 --2- 172.21.15.37:0/3812901059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b8108a60 0x7f59b8108e40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 --2- 172.21.15.37:0/3812901059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806bbb0 0x7f59b810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 --2- 172.21.15.37:0/3812901059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b806b1f0 0x7f59b806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 -- 172.21.15.37:0/3812901059 >> 172.21.15.37:0/3812901059 conn(0x7f59b80789e0 msgr2=0x7f59b8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 -- 172.21.15.37:0/3812901059 shutdown_connections 2024-04-23T18:19:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.602+0000 7f59bfd50700 1 -- 172.21.15.37:0/3812901059 wait complete. 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 Processor -- start 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 -- start start 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806b1f0 0x7f59b81019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b806bbb0 0x7f59b8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8108a60 0x7f59b8102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112c60 con 0x7f59b806bbb0 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112ae0 con 0x7f59b8108a60 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bfd50700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8112de0 con 0x7f59b806b1f0 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bd2eb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b806bbb0 0x7f59b8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bd2eb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b806bbb0 0x7f59b8101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50452/0 (socket says 172.21.15.37:50452) 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bd2eb700 1 -- 172.21.15.37:0/1443684019 learned_addr learned my addr 172.21.15.37:0/1443684019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:45.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bdaec700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806b1f0 0x7f59b81019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59be2ed700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8108a60 0x7f59b8102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bdaec700 1 -- 172.21.15.37:0/1443684019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8108a60 msgr2=0x7f59b8102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bdaec700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8108a60 0x7f59b8102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bdaec700 1 -- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b806bbb0 msgr2=0x7f59b8101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bdaec700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b806bbb0 0x7f59b8101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.603+0000 7f59bdaec700 1 -- 172.21.15.37:0/1443684019 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b81a7690 con 0x7f59b806b1f0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.604+0000 7f59bdaec700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806b1f0 0x7f59b81019b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59b40059d0 tx=0x7f59b40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.604+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4027070 con 0x7f59b806b1f0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.604+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b81a7920 con 0x7f59b806b1f0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.604+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b4022470 con 0x7f59b806b1f0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.604+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4017650 con 0x7f59b806b1f0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.604+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a7dd0 con 0x7f59b806b1f0 2024-04-23T18:19:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.605+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b810b060 con 0x7f59b806b1f0 2024-04-23T18:19:45.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.606+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59b402a070 con 0x7f59b806b1f0 2024-04-23T18:19:45.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.607+0000 7f59aeffd700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59a40a3cd0 0x7f59a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.607+0000 7f59bd2eb700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59a40a3cd0 0x7f59a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.607+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f59b40d1630 con 0x7f59b806b1f0 2024-04-23T18:19:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.608+0000 7f59bd2eb700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59a40a3cd0 0x7f59a40a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59b80790c0 tx=0x7f59a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.611+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59b4013070 con 0x7f59b806b1f0 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 -- 172.21.15.37:0/4113508618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df41063f0 msgr2=0x7f0df410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/4113508618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df41063f0 0x7f0df410dfc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00b0d0 tx=0x7f0dec00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 -- 172.21.15.37:0/4113508618 shutdown_connections 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/4113508618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df41063f0 0x7f0df410dfc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/4113508618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df4101360 0x7f0df4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/4113508618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101d10 0x7f0df4105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 -- 172.21.15.37:0/4113508618 >> 172.21.15.37:0/4113508618 conn(0x7f0df4077fb0 msgr2=0x7f0df40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.506+0000 7f0dfab5f700 1 -- 172.21.15.37:0/4113508618 shutdown_connections 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 -- 172.21.15.37:0/4113508618 wait complete. 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 Processor -- start 2024-04-23T18:19:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 -- start start 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101360 0x7f0df419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 0x7f0df419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df41063f0 0x7f0df41a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41112b0 con 0x7f0df4101d10 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4111130 con 0x7f0df4101360 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0dfab5f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4111430 con 0x7f0df41063f0 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0debfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 0x7f0df419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0debfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 0x7f0df419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50434/0 (socket says 172.21.15.37:50434) 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.507+0000 7f0debfff700 1 -- 172.21.15.37:0/1336959968 learned_addr learned my addr 172.21.15.37:0/1336959968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0debfff700 1 -- 172.21.15.37:0/1336959968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df41063f0 msgr2=0x7f0df41a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0df88fb700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101360 0x7f0df419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0df90fc700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df41063f0 0x7f0df41a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0debfff700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df41063f0 0x7f0df41a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0debfff700 1 -- 172.21.15.37:0/1336959968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101360 msgr2=0x7f0df419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0debfff700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101360 0x7f0df419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0debfff700 1 -- 172.21.15.37:0/1336959968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df41a55d0 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0df88fb700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101360 0x7f0df419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0df90fc700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df41063f0 0x7f0df41a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0debfff700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 0x7f0df419a550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c8f0 tx=0x7f0de400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4007b00 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de4007ca0 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df41a57a0 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4018660 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.508+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df41a5d00 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.509+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df41027f0 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.510+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0de400f450 con 0x7f0df4101d10 2024-04-23T18:19:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.511+0000 7f0de9ffb700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0ddc0a3d50 0x7f0ddc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.511+0000 7f0df88fb700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0ddc0a3d50 0x7f0ddc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.511+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0de40d0490 con 0x7f0df4101d10 2024-04-23T18:19:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.512+0000 7f0df88fb700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0ddc0a3d50 0x7f0ddc0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005fd0 tx=0x7f0df0005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.516+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0de4014030 con 0x7f0df4101d10 2024-04-23T18:19:45.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.727+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0df41a5fc0 con 0x7f0df4101d10 2024-04-23T18:19:45.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.727+0000 7f0de9ffb700 1 -- 172.21.15.37:0/1336959968 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0de409cd00 con 0x7f0df4101d10 2024-04-23T18:19:45.720 INFO:teuthology.orchestra.run.smithi037.stdout:201863462963 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0ddc0a3d50 msgr2=0x7f0ddc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0ddc0a3d50 0x7f0ddc0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005fd0 tx=0x7f0df0005dc0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 msgr2=0x7f0df419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 0x7f0df419a550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c8f0 tx=0x7f0de400ccb0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 shutdown_connections 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0ddc0a3d50 0x7f0ddc0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df41063f0 0x7f0df41a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df4101360 0x7f0df419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 --2- 172.21.15.37:0/1336959968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df4101d10 0x7f0df419a550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 >> 172.21.15.37:0/1336959968 conn(0x7f0df4077fb0 msgr2=0x7f0df40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 shutdown_connections 2024-04-23T18:19:45.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.729+0000 7f0dfab5f700 1 -- 172.21.15.37:0/1336959968 wait complete. 2024-04-23T18:19:45.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.827+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f59b8102e90 con 0x7f59b806b1f0 2024-04-23T18:19:45.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.827+0000 7f59aeffd700 1 -- 172.21.15.37:0/1443684019 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f59b409dff0 con 0x7f59b806b1f0 2024-04-23T18:19:45.821 INFO:teuthology.orchestra.run.smithi037.stdout:356482285595 2024-04-23T18:19:45.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59a40a3cd0 msgr2=0x7f59a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59a40a3cd0 0x7f59a40a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59b80790c0 tx=0x7f59a8009260 comp rx=0 tx=0).stop 2024-04-23T18:19:45.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806b1f0 msgr2=0x7f59b81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:45.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806b1f0 0x7f59b81019b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59b40059d0 tx=0x7f59b40047b0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 shutdown_connections 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f59a40a3cd0 0x7f59a40a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8108a60 0x7f59b8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59b806b1f0 0x7f59b81019b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 --2- 172.21.15.37:0/1443684019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b806bbb0 0x7f59b8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 >> 172.21.15.37:0/1443684019 conn(0x7f59b80789e0 msgr2=0x7f59b80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 shutdown_connections 2024-04-23T18:19:45.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:45.830+0000 7f59bfd50700 1 -- 172.21.15.37:0/1443684019 wait complete. 2024-04-23T18:19:46.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:45 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1336959968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:19:46.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:45 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1336959968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:19:46.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:45 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1336959968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:19:46.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285588 got 356482285595 for osd.11 2024-04-23T18:19:46.508 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:46.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462963 for osd.6 2024-04-23T18:19:46.920 DEBUG:teuthology.parallel:result is None 2024-04-23T18:19:46.920 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T18:19:46.920 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:19:47.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:46 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1443684019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:19:47.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:46 smithi172 ceph-mon[32456]: pgmap v323: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 7.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:47.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:46 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1443684019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:19:47.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:46 smithi173 ceph-mon[32906]: pgmap v323: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 7.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:47.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:46 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1443684019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:19:47.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:46 smithi037 ceph-mon[29404]: pgmap v323: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 7.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:47.289 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:48.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:47 smithi037 ceph-mon[29404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-23T18:19:48.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:47 smithi037 ceph-mon[29404]: Cluster is now healthy 2024-04-23T18:19:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:47 smithi172 ceph-mon[32456]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-23T18:19:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:47 smithi172 ceph-mon[32456]: Cluster is now healthy 2024-04-23T18:19:48.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:47 smithi173 ceph-mon[32906]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-23T18:19:48.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:47 smithi173 ceph-mon[32906]: Cluster is now healthy 2024-04-23T18:19:48.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 -- 172.21.15.37:0/1018768633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b006b0f0 msgr2=0x7fc7b010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:48.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 --2- 172.21.15.37:0/1018768633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b006b0f0 0x7fc7b010f900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800b0a0 tx=0x7fc7a800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 -- 172.21.15.37:0/1018768633 shutdown_connections 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 --2- 172.21.15.37:0/1018768633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b006b0f0 0x7fc7b010f900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 --2- 172.21.15.37:0/1018768633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7b0108a30 0x7fc7b0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 --2- 172.21.15.37:0/1018768633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7b006a750 0x7fc7b006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 -- 172.21.15.37:0/1018768633 >> 172.21.15.37:0/1018768633 conn(0x7fc7b00788f0 msgr2=0x7fc7b0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 -- 172.21.15.37:0/1018768633 shutdown_connections 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.899+0000 7fc7b701f700 1 -- 172.21.15.37:0/1018768633 wait complete. 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 Processor -- start 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 -- start start 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7b006a750 0x7fc7b0199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7b006b0f0 0x7fc7b0199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 0x7fc7b019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b0112c20 con 0x7fc7b0108a30 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b0112aa0 con 0x7fc7b006b0f0 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b701f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b0112da0 con 0x7fc7b006a750 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b55bc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 0x7fc7b019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b55bc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 0x7fc7b019d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50466/0 (socket says 172.21.15.37:50466) 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b55bc700 1 -- 172.21.15.37:0/3439590319 learned_addr learned my addr 172.21.15.37:0/3439590319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7b4dbb700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7b006a750 0x7fc7b0199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.900+0000 7fc7affff700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7b006b0f0 0x7fc7b0199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b55bc700 1 -- 172.21.15.37:0/3439590319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7b006a750 msgr2=0x7fc7b0199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b55bc700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7b006a750 0x7fc7b0199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b55bc700 1 -- 172.21.15.37:0/3439590319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7b006b0f0 msgr2=0x7fc7b0199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b55bc700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7b006b0f0 0x7fc7b0199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b55bc700 1 -- 172.21.15.37:0/3439590319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b019e010 con 0x7fc7b0108a30 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b55bc700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 0x7fc7b019d8d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800b070 tx=0x7fc7a800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8021070 con 0x7fc7b0108a30 2024-04-23T18:19:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b019e2a0 con 0x7fc7b0108a30 2024-04-23T18:19:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a8007c80 con 0x7fc7b0108a30 2024-04-23T18:19:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8005440 con 0x7fc7b0108a30 2024-04-23T18:19:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.901+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b01a5b80 con 0x7fc7b0108a30 2024-04-23T18:19:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.902+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b0069730 con 0x7fc7b0108a30 2024-04-23T18:19:48.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.904+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7a8024070 con 0x7fc7b0108a30 2024-04-23T18:19:48.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.904+0000 7fc7adffb700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc7a00a3cd0 0x7fc7a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.905+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc7a80d1770 con 0x7fc7b0108a30 2024-04-23T18:19:48.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.905+0000 7fc7b4dbb700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc7a00a3cd0 0x7fc7a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.905+0000 7fc7b4dbb700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc7a00a3cd0 0x7fc7a00a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400abd0 tx=0x7fc7a400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:48.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:48.908+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7a809e070 con 0x7fc7b0108a30 2024-04-23T18:19:49.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.129+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7b019a360 con 0x7fc7a00a3cd0 2024-04-23T18:19:49.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.137+0000 7fc7adffb700 1 -- 172.21.15.37:0/3439590319 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258729 (secure 0 0 0) 0x7fc7b019a360 con 0x7fc7a00a3cd0 2024-04-23T18:19:49.134 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc7a00a3cd0 msgr2=0x7fc7a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc7a00a3cd0 0x7fc7a00a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400abd0 tx=0x7fc7a400a380 comp rx=0 tx=0).stop 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 msgr2=0x7fc7b019d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 0x7fc7b019d8d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800b070 tx=0x7fc7a800ce50 comp rx=0 tx=0).stop 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 shutdown_connections 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc7a00a3cd0 0x7fc7a00a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b0108a30 0x7fc7b019d8d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7b006a750 0x7fc7b0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 --2- 172.21.15.37:0/3439590319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7b006b0f0 0x7fc7b0199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.140+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 >> 172.21.15.37:0/3439590319 conn(0x7fc7b00788f0 msgr2=0x7fc7b00ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.141+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 shutdown_connections 2024-04-23T18:19:49.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:49.141+0000 7fc7b701f700 1 -- 172.21.15.37:0/3439590319 wait complete. 2024-04-23T18:19:49.138 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:19:49.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:48 smithi172 ceph-mon[32456]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3 op/s; 7.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:49.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:48 smithi173 ceph-mon[32906]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3 op/s; 7.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:49.360 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:48 smithi037 ceph-mon[29404]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3 op/s; 7.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:49.708 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-23T18:19:47.878139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":547,"num_read_kb":393,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1241933,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375412,"kb_used_data":13692,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566412,"statfs":{"total":1151940427776,"available":1151556005888,"internally_reserved":0,"allocated":14020608,"data_stored":7635108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61197,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002200"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689574+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:19:41.689574+0000","last_peered":"2024-04-23T18:19:41.689574+0000","last_clean":"2024-04-23T18:19:41.689574+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:19:41.689574+0000","last_undegraded":"2024-04-23T18:19:41.689574+0000","last_fullsized":"2024-04-23T18:19:41.689574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002051+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:19:42.002051+0000","last_peered":"2024-04-23T18:19:42.002051+0000","last_clean":"2024-04-23T18:19:42.002051+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:19:42.002051+0000","last_undegraded":"2024-04-23T18:19:42.002051+0000","last_fullsized":"2024-04-23T18:19:42.002051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002082+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:19:42.002082+0000","last_peered":"2024-04-23T18:19:42.002082+0000","last_clean":"2024-04-23T18:19:42.002082+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:19:42.002082+0000","last_undegraded":"2024-04-23T18:19:42.002082+0000","last_fullsized":"2024-04-23T18:19:42.002082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.695047+0000","last_change":"2024-04-23T18:19:41.694639+0000","last_active":"2024-04-23T18:19:41.695047+0000","last_peered":"2024-04-23T18:19:41.695047+0000","last_clean":"2024-04-23T18:19:41.695047+0000","last_became_active":"2024-04-23T18:19:41.694291+0000","last_became_peered":"2024-04-23T18:19:41.694291+0000","last_unstale":"2024-04-23T18:19:41.695047+0000","last_undegraded":"2024-04-23T18:19:41.695047+0000","last_fullsized":"2024-04-23T18:19:41.695047+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:44.268737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692247+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:19:41.692247+0000","last_peered":"2024-04-23T18:19:41.692247+0000","last_clean":"2024-04-23T18:19:41.692247+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:19:41.692247+0000","last_undegraded":"2024-04-23T18:19:41.692247+0000","last_fullsized":"2024-04-23T18:19:41.692247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689447+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:19:41.689447+0000","last_peered":"2024-04-23T18:19:41.689447+0000","last_clean":"2024-04-23T18:19:41.689447+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:19:41.689447+0000","last_undegraded":"2024-04-23T18:19:41.689447+0000","last_fullsized":"2024-04-23T18:19:41.689447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689135+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:19:41.689135+0000","last_peered":"2024-04-23T18:19:41.689135+0000","last_clean":"2024-04-23T18:19:41.689135+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:19:41.689135+0000","last_undegraded":"2024-04-23T18:19:41.689135+0000","last_fullsized":"2024-04-23T18:19:41.689135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687857+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:19:41.687857+0000","last_peered":"2024-04-23T18:19:41.687857+0000","last_clean":"2024-04-23T18:19:41.687857+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:19:41.687857+0000","last_undegraded":"2024-04-23T18:19:41.687857+0000","last_fullsized":"2024-04-23T18:19:41.687857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687228+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:19:41.687228+0000","last_peered":"2024-04-23T18:19:41.687228+0000","last_clean":"2024-04-23T18:19:41.687228+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:19:41.687228+0000","last_undegraded":"2024-04-23T18:19:41.687228+0000","last_fullsized":"2024-04-23T18:19:41.687228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688707+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:19:41.688707+0000","last_peered":"2024-04-23T18:19:41.688707+0000","last_clean":"2024-04-23T18:19:41.688707+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:19:41.688707+0000","last_undegraded":"2024-04-23T18:19:41.688707+0000","last_fullsized":"2024-04-23T18:19:41.688707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690352+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:19:41.690352+0000","last_peered":"2024-04-23T18:19:41.690352+0000","last_clean":"2024-04-23T18:19:41.690352+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:19:41.690352+0000","last_undegraded":"2024-04-23T18:19:41.690352+0000","last_fullsized":"2024-04-23T18:19:41.690352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.930064+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:19:41.930064+0000","last_peered":"2024-04-23T18:19:41.930064+0000","last_clean":"2024-04-23T18:19:41.930064+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:19:41.930064+0000","last_undegraded":"2024-04-23T18:19:41.930064+0000","last_fullsized":"2024-04-23T18:19:41.930064+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686269+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:19:41.686269+0000","last_peered":"2024-04-23T18:19:41.686269+0000","last_clean":"2024-04-23T18:19:41.686269+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:19:41.686269+0000","last_undegraded":"2024-04-23T18:19:41.686269+0000","last_fullsized":"2024-04-23T18:19:41.686269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.001902+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:19:42.001902+0000","last_peered":"2024-04-23T18:19:42.001902+0000","last_clean":"2024-04-23T18:19:42.001902+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:19:42.001902+0000","last_undegraded":"2024-04-23T18:19:42.001902+0000","last_fullsized":"2024-04-23T18:19:42.001902+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.001899+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:19:42.001899+0000","last_peered":"2024-04-23T18:19:42.001899+0000","last_clean":"2024-04-23T18:19:42.001899+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:19:42.001899+0000","last_undegraded":"2024-04-23T18:19:42.001899+0000","last_fullsized":"2024-04-23T18:19:42.001899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686059+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:19:41.686059+0000","last_peered":"2024-04-23T18:19:41.686059+0000","last_clean":"2024-04-23T18:19:41.686059+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:19:41.686059+0000","last_undegraded":"2024-04-23T18:19:41.686059+0000","last_fullsized":"2024-04-23T18:19:41.686059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689690+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:19:41.689690+0000","last_peered":"2024-04-23T18:19:41.689690+0000","last_clean":"2024-04-23T18:19:41.689690+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:19:41.689690+0000","last_undegraded":"2024-04-23T18:19:41.689690+0000","last_fullsized":"2024-04-23T18:19:41.689690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688150+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:19:41.688150+0000","last_peered":"2024-04-23T18:19:41.688150+0000","last_clean":"2024-04-23T18:19:41.688150+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:19:41.688150+0000","last_undegraded":"2024-04-23T18:19:41.688150+0000","last_fullsized":"2024-04-23T18:19:41.688150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690857+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:19:41.690857+0000","last_peered":"2024-04-23T18:19:41.690857+0000","last_clean":"2024-04-23T18:19:41.690857+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:19:41.690857+0000","last_undegraded":"2024-04-23T18:19:41.690857+0000","last_fullsized":"2024-04-23T18:19:41.690857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688368+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:19:41.688368+0000","last_peered":"2024-04-23T18:19:41.688368+0000","last_clean":"2024-04-23T18:19:41.688368+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:19:41.688368+0000","last_undegraded":"2024-04-23T18:19:41.688368+0000","last_fullsized":"2024-04-23T18:19:41.688368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688217+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:19:41.688217+0000","last_peered":"2024-04-23T18:19:41.688217+0000","last_clean":"2024-04-23T18:19:41.688217+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:19:41.688217+0000","last_undegraded":"2024-04-23T18:19:41.688217+0000","last_fullsized":"2024-04-23T18:19:41.688217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688479+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:19:41.688479+0000","last_peered":"2024-04-23T18:19:41.688479+0000","last_clean":"2024-04-23T18:19:41.688479+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:19:41.688479+0000","last_undegraded":"2024-04-23T18:19:41.688479+0000","last_fullsized":"2024-04-23T18:19:41.688479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002174+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:19:42.002174+0000","last_peered":"2024-04-23T18:19:42.002174+0000","last_clean":"2024-04-23T18:19:42.002174+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:19:42.002174+0000","last_undegraded":"2024-04-23T18:19:42.002174+0000","last_fullsized":"2024-04-23T18:19:42.002174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692597+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:19:41.692597+0000","last_peered":"2024-04-23T18:19:41.692597+0000","last_clean":"2024-04-23T18:19:41.692597+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:19:41.692597+0000","last_undegraded":"2024-04-23T18:19:41.692597+0000","last_fullsized":"2024-04-23T18:19:41.692597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686361+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:19:41.686361+0000","last_peered":"2024-04-23T18:19:41.686361+0000","last_clean":"2024-04-23T18:19:41.686361+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:19:41.686361+0000","last_undegraded":"2024-04-23T18:19:41.686361+0000","last_fullsized":"2024-04-23T18:19:41.686361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.693677+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:19:41.693677+0000","last_peered":"2024-04-23T18:19:41.693677+0000","last_clean":"2024-04-23T18:19:41.693677+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:19:41.693677+0000","last_undegraded":"2024-04-23T18:19:41.693677+0000","last_fullsized":"2024-04-23T18:19:41.693677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686834+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:19:41.686834+0000","last_peered":"2024-04-23T18:19:41.686834+0000","last_clean":"2024-04-23T18:19:41.686834+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:19:41.686834+0000","last_undegraded":"2024-04-23T18:19:41.686834+0000","last_fullsized":"2024-04-23T18:19:41.686834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:43.191144+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:19:43.191144+0000","last_peered":"2024-04-23T18:19:43.191144+0000","last_clean":"2024-04-23T18:19:43.191144+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:19:43.191144+0000","last_undegraded":"2024-04-23T18:19:43.191144+0000","last_fullsized":"2024-04-23T18:19:43.191144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692687+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:19:41.692687+0000","last_peered":"2024-04-23T18:19:41.692687+0000","last_clean":"2024-04-23T18:19:41.692687+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:19:41.692687+0000","last_undegraded":"2024-04-23T18:19:41.692687+0000","last_fullsized":"2024-04-23T18:19:41.692687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.933594+0000","last_change":"2024-04-23T18:19:41.933594+0000","last_active":"2024-04-23T18:19:41.933594+0000","last_peered":"2024-04-23T18:19:41.933594+0000","last_clean":"2024-04-23T18:19:41.933594+0000","last_became_active":"2024-04-23T18:19:41.930226+0000","last_became_peered":"2024-04-23T18:19:41.930226+0000","last_unstale":"2024-04-23T18:19:41.933594+0000","last_undegraded":"2024-04-23T18:19:41.933594+0000","last_fullsized":"2024-04-23T18:19:41.933594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:57.183719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.694345+0000","last_change":"2024-04-23T18:19:41.694345+0000","last_active":"2024-04-23T18:19:41.694345+0000","last_peered":"2024-04-23T18:19:41.694345+0000","last_clean":"2024-04-23T18:19:41.694345+0000","last_became_active":"2024-04-23T18:19:41.693912+0000","last_became_peered":"2024-04-23T18:19:41.693912+0000","last_unstale":"2024-04-23T18:19:41.694345+0000","last_undegraded":"2024-04-23T18:19:41.694345+0000","last_fullsized":"2024-04-23T18:19:41.694345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:12:07.153403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686049+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:19:41.686049+0000","last_peered":"2024-04-23T18:19:41.686049+0000","last_clean":"2024-04-23T18:19:41.686049+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:19:41.686049+0000","last_undegraded":"2024-04-23T18:19:41.686049+0000","last_fullsized":"2024-04-23T18:19:41.686049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687916+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:19:41.687916+0000","last_peered":"2024-04-23T18:19:41.687916+0000","last_clean":"2024-04-23T18:19:41.687916+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:19:41.687916+0000","last_undegraded":"2024-04-23T18:19:41.687916+0000","last_fullsized":"2024-04-23T18:19:41.687916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686238+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:19:41.686238+0000","last_peered":"2024-04-23T18:19:41.686238+0000","last_clean":"2024-04-23T18:19:41.686238+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:19:41.686238+0000","last_undegraded":"2024-04-23T18:19:41.686238+0000","last_fullsized":"2024-04-23T18:19:41.686238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688522+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:19:41.688522+0000","last_peered":"2024-04-23T18:19:41.688522+0000","last_clean":"2024-04-23T18:19:41.688522+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:19:41.688522+0000","last_undegraded":"2024-04-23T18:19:41.688522+0000","last_fullsized":"2024-04-23T18:19:41.688522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":87,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:43.190514+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:19:43.190514+0000","last_peered":"2024-04-23T18:19:43.190514+0000","last_clean":"2024-04-23T18:19:43.190514+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:19:43.190514+0000","last_undegraded":"2024-04-23T18:19:43.190514+0000","last_fullsized":"2024-04-23T18:19:43.190514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687560+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:19:41.687560+0000","last_peered":"2024-04-23T18:19:41.687560+0000","last_clean":"2024-04-23T18:19:41.687560+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:19:41.687560+0000","last_undegraded":"2024-04-23T18:19:41.687560+0000","last_fullsized":"2024-04-23T18:19:41.687560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690065+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:19:41.690065+0000","last_peered":"2024-04-23T18:19:41.690065+0000","last_clean":"2024-04-23T18:19:41.690065+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:19:41.690065+0000","last_undegraded":"2024-04-23T18:19:41.690065+0000","last_fullsized":"2024-04-23T18:19:41.690065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002134+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:19:42.002134+0000","last_peered":"2024-04-23T18:19:42.002134+0000","last_clean":"2024-04-23T18:19:42.002134+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:19:42.002134+0000","last_undegraded":"2024-04-23T18:19:42.002134+0000","last_fullsized":"2024-04-23T18:19:42.002134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686989+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:19:41.686989+0000","last_peered":"2024-04-23T18:19:41.686989+0000","last_clean":"2024-04-23T18:19:41.686989+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:19:41.686989+0000","last_undegraded":"2024-04-23T18:19:41.686989+0000","last_fullsized":"2024-04-23T18:19:41.686989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.694590+0000","last_change":"2024-04-23T18:19:41.694590+0000","last_active":"2024-04-23T18:19:41.694590+0000","last_peered":"2024-04-23T18:19:41.694590+0000","last_clean":"2024-04-23T18:19:41.694590+0000","last_became_active":"2024-04-23T18:19:41.694163+0000","last_became_peered":"2024-04-23T18:19:41.694163+0000","last_unstale":"2024-04-23T18:19:41.694590+0000","last_undegraded":"2024-04-23T18:19:41.694590+0000","last_fullsized":"2024-04-23T18:19:41.694590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:28.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:43.937085+0000","last_change":"2024-04-23T18:19:41.940140+0000","last_active":"2024-04-23T18:19:43.937085+0000","last_peered":"2024-04-23T18:19:43.937085+0000","last_clean":"2024-04-23T18:19:43.937085+0000","last_became_active":"2024-04-23T18:19:41.931221+0000","last_became_peered":"2024-04-23T18:19:41.931221+0000","last_unstale":"2024-04-23T18:19:43.937085+0000","last_undegraded":"2024-04-23T18:19:43.937085+0000","last_fullsized":"2024-04-23T18:19:43.937085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:40.122956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":124,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692395+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:19:41.692395+0000","last_peered":"2024-04-23T18:19:41.692395+0000","last_clean":"2024-04-23T18:19:41.692395+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:19:41.692395+0000","last_undegraded":"2024-04-23T18:19:41.692395+0000","last_fullsized":"2024-04-23T18:19:41.692395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.700269+0000","last_change":"2024-04-23T18:19:41.699880+0000","last_active":"2024-04-23T18:19:41.700269+0000","last_peered":"2024-04-23T18:19:41.700269+0000","last_clean":"2024-04-23T18:19:41.700269+0000","last_became_active":"2024-04-23T18:19:41.691650+0000","last_became_peered":"2024-04-23T18:19:41.691650+0000","last_unstale":"2024-04-23T18:19:41.700269+0000","last_undegraded":"2024-04-23T18:19:41.700269+0000","last_fullsized":"2024-04-23T18:19:41.700269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:28.166844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.685994+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:19:41.685994+0000","last_peered":"2024-04-23T18:19:41.685994+0000","last_clean":"2024-04-23T18:19:41.685994+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:19:41.685994+0000","last_undegraded":"2024-04-23T18:19:41.685994+0000","last_fullsized":"2024-04-23T18:19:41.685994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686376+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:19:41.686376+0000","last_peered":"2024-04-23T18:19:41.686376+0000","last_clean":"2024-04-23T18:19:41.686376+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:19:41.686376+0000","last_undegraded":"2024-04-23T18:19:41.686376+0000","last_fullsized":"2024-04-23T18:19:41.686376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689253+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:19:41.689253+0000","last_peered":"2024-04-23T18:19:41.689253+0000","last_clean":"2024-04-23T18:19:41.689253+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:19:41.689253+0000","last_undegraded":"2024-04-23T18:19:41.689253+0000","last_fullsized":"2024-04-23T18:19:41.689253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:44.191870+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:19:44.191870+0000","last_peered":"2024-04-23T18:19:44.191870+0000","last_clean":"2024-04-23T18:19:44.191870+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:19:44.191870+0000","last_undegraded":"2024-04-23T18:19:44.191870+0000","last_fullsized":"2024-04-23T18:19:44.191870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688720+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:19:41.688720+0000","last_peered":"2024-04-23T18:19:41.688720+0000","last_clean":"2024-04-23T18:19:41.688720+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:19:41.688720+0000","last_undegraded":"2024-04-23T18:19:41.688720+0000","last_fullsized":"2024-04-23T18:19:41.688720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692312+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:19:41.692312+0000","last_peered":"2024-04-23T18:19:41.692312+0000","last_clean":"2024-04-23T18:19:41.692312+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:19:41.692312+0000","last_undegraded":"2024-04-23T18:19:41.692312+0000","last_fullsized":"2024-04-23T18:19:41.692312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686403+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:19:41.686403+0000","last_peered":"2024-04-23T18:19:41.686403+0000","last_clean":"2024-04-23T18:19:41.686403+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:19:41.686403+0000","last_undegraded":"2024-04-23T18:19:41.686403+0000","last_fullsized":"2024-04-23T18:19:41.686403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692281+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:19:41.692281+0000","last_peered":"2024-04-23T18:19:41.692281+0000","last_clean":"2024-04-23T18:19:41.692281+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:19:41.692281+0000","last_undegraded":"2024-04-23T18:19:41.692281+0000","last_fullsized":"2024-04-23T18:19:41.692281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689429+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:19:41.689429+0000","last_peered":"2024-04-23T18:19:41.689429+0000","last_clean":"2024-04-23T18:19:41.689429+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:19:41.689429+0000","last_undegraded":"2024-04-23T18:19:41.689429+0000","last_fullsized":"2024-04-23T18:19:41.689429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688511+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:19:41.688511+0000","last_peered":"2024-04-23T18:19:41.688511+0000","last_clean":"2024-04-23T18:19:41.688511+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:19:41.688511+0000","last_undegraded":"2024-04-23T18:19:41.688511+0000","last_fullsized":"2024-04-23T18:19:41.688511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688442+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:19:41.688442+0000","last_peered":"2024-04-23T18:19:41.688442+0000","last_clean":"2024-04-23T18:19:41.688442+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:19:41.688442+0000","last_undegraded":"2024-04-23T18:19:41.688442+0000","last_fullsized":"2024-04-23T18:19:41.688442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689502+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:19:41.689502+0000","last_peered":"2024-04-23T18:19:41.689502+0000","last_clean":"2024-04-23T18:19:41.689502+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:19:41.689502+0000","last_undegraded":"2024-04-23T18:19:41.689502+0000","last_fullsized":"2024-04-23T18:19:41.689502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.003042+0000","last_change":"2024-04-23T18:19:42.003042+0000","last_active":"2024-04-23T18:19:42.003042+0000","last_peered":"2024-04-23T18:19:42.003042+0000","last_clean":"2024-04-23T18:19:42.003042+0000","last_became_active":"2024-04-23T18:19:42.002728+0000","last_became_peered":"2024-04-23T18:19:42.002728+0000","last_unstale":"2024-04-23T18:19:42.003042+0000","last_undegraded":"2024-04-23T18:19:42.003042+0000","last_fullsized":"2024-04-23T18:19:42.003042+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:54:33.401918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686407+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:19:41.686407+0000","last_peered":"2024-04-23T18:19:41.686407+0000","last_clean":"2024-04-23T18:19:41.686407+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:19:41.686407+0000","last_undegraded":"2024-04-23T18:19:41.686407+0000","last_fullsized":"2024-04-23T18:19:41.686407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:43.189849+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:19:43.189849+0000","last_peered":"2024-04-23T18:19:43.189849+0000","last_clean":"2024-04-23T18:19:43.189849+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:19:43.189849+0000","last_undegraded":"2024-04-23T18:19:43.189849+0000","last_fullsized":"2024-04-23T18:19:43.189849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.691129+0000","last_change":"2024-04-23T18:19:41.691129+0000","last_active":"2024-04-23T18:19:41.691129+0000","last_peered":"2024-04-23T18:19:41.691129+0000","last_clean":"2024-04-23T18:19:41.691129+0000","last_became_active":"2024-04-23T18:19:41.690710+0000","last_became_peered":"2024-04-23T18:19:41.690710+0000","last_unstale":"2024-04-23T18:19:41.691129+0000","last_undegraded":"2024-04-23T18:19:41.691129+0000","last_fullsized":"2024-04-23T18:19:41.691129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:51.914794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.930135+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:19:41.930135+0000","last_peered":"2024-04-23T18:19:41.930135+0000","last_clean":"2024-04-23T18:19:41.930135+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:19:41.930135+0000","last_undegraded":"2024-04-23T18:19:41.930135+0000","last_fullsized":"2024-04-23T18:19:41.930135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689005+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:19:41.689005+0000","last_peered":"2024-04-23T18:19:41.689005+0000","last_clean":"2024-04-23T18:19:41.689005+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:19:41.689005+0000","last_undegraded":"2024-04-23T18:19:41.689005+0000","last_fullsized":"2024-04-23T18:19:41.689005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.685897+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:19:41.685897+0000","last_peered":"2024-04-23T18:19:41.685897+0000","last_clean":"2024-04-23T18:19:41.685897+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:19:41.685897+0000","last_undegraded":"2024-04-23T18:19:41.685897+0000","last_fullsized":"2024-04-23T18:19:41.685897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689084+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:19:41.689084+0000","last_peered":"2024-04-23T18:19:41.689084+0000","last_clean":"2024-04-23T18:19:41.689084+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:19:41.689084+0000","last_undegraded":"2024-04-23T18:19:41.689084+0000","last_fullsized":"2024-04-23T18:19:41.689084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692563+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:19:41.692563+0000","last_peered":"2024-04-23T18:19:41.692563+0000","last_clean":"2024-04-23T18:19:41.692563+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:19:41.692563+0000","last_undegraded":"2024-04-23T18:19:41.692563+0000","last_fullsized":"2024-04-23T18:19:41.692563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686127+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:19:41.686127+0000","last_peered":"2024-04-23T18:19:41.686127+0000","last_clean":"2024-04-23T18:19:41.686127+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:19:41.686127+0000","last_undegraded":"2024-04-23T18:19:41.686127+0000","last_fullsized":"2024-04-23T18:19:41.686127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688371+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:19:41.688371+0000","last_peered":"2024-04-23T18:19:41.688371+0000","last_clean":"2024-04-23T18:19:41.688371+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:19:41.688371+0000","last_undegraded":"2024-04-23T18:19:41.688371+0000","last_fullsized":"2024-04-23T18:19:41.688371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688615+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:19:41.688615+0000","last_peered":"2024-04-23T18:19:41.688615+0000","last_clean":"2024-04-23T18:19:41.688615+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:19:41.688615+0000","last_undegraded":"2024-04-23T18:19:41.688615+0000","last_fullsized":"2024-04-23T18:19:41.688615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.929678+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:19:41.929678+0000","last_peered":"2024-04-23T18:19:41.929678+0000","last_clean":"2024-04-23T18:19:41.929678+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:19:41.929678+0000","last_undegraded":"2024-04-23T18:19:41.929678+0000","last_fullsized":"2024-04-23T18:19:41.929678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002179+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:19:42.002179+0000","last_peered":"2024-04-23T18:19:42.002179+0000","last_clean":"2024-04-23T18:19:42.002179+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:19:42.002179+0000","last_undegraded":"2024-04-23T18:19:42.002179+0000","last_fullsized":"2024-04-23T18:19:42.002179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686081+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:19:41.686081+0000","last_peered":"2024-04-23T18:19:41.686081+0000","last_clean":"2024-04-23T18:19:41.686081+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:19:41.686081+0000","last_undegraded":"2024-04-23T18:19:41.686081+0000","last_fullsized":"2024-04-23T18:19:41.686081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692397+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:19:41.692397+0000","last_peered":"2024-04-23T18:19:41.692397+0000","last_clean":"2024-04-23T18:19:41.692397+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:19:41.692397+0000","last_undegraded":"2024-04-23T18:19:41.692397+0000","last_fullsized":"2024-04-23T18:19:41.692397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688264+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:19:41.688264+0000","last_peered":"2024-04-23T18:19:41.688264+0000","last_clean":"2024-04-23T18:19:41.688264+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:19:41.688264+0000","last_undegraded":"2024-04-23T18:19:41.688264+0000","last_fullsized":"2024-04-23T18:19:41.688264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689190+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:19:41.689190+0000","last_peered":"2024-04-23T18:19:41.689190+0000","last_clean":"2024-04-23T18:19:41.689190+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:19:41.689190+0000","last_undegraded":"2024-04-23T18:19:41.689190+0000","last_fullsized":"2024-04-23T18:19:41.689190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690975+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:19:41.690975+0000","last_peered":"2024-04-23T18:19:41.690975+0000","last_clean":"2024-04-23T18:19:41.690975+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:19:41.690975+0000","last_undegraded":"2024-04-23T18:19:41.690975+0000","last_fullsized":"2024-04-23T18:19:41.690975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688094+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:19:41.688094+0000","last_peered":"2024-04-23T18:19:41.688094+0000","last_clean":"2024-04-23T18:19:41.688094+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:19:41.688094+0000","last_undegraded":"2024-04-23T18:19:41.688094+0000","last_fullsized":"2024-04-23T18:19:41.688094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692137+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:19:41.692137+0000","last_peered":"2024-04-23T18:19:41.692137+0000","last_clean":"2024-04-23T18:19:41.692137+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:19:41.692137+0000","last_undegraded":"2024-04-23T18:19:41.692137+0000","last_fullsized":"2024-04-23T18:19:41.692137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689640+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:19:41.689640+0000","last_peered":"2024-04-23T18:19:41.689640+0000","last_clean":"2024-04-23T18:19:41.689640+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:19:41.689640+0000","last_undegraded":"2024-04-23T18:19:41.689640+0000","last_fullsized":"2024-04-23T18:19:41.689640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690835+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:19:41.690835+0000","last_peered":"2024-04-23T18:19:41.690835+0000","last_clean":"2024-04-23T18:19:41.690835+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:19:41.690835+0000","last_undegraded":"2024-04-23T18:19:41.690835+0000","last_fullsized":"2024-04-23T18:19:41.690835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690468+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:19:41.690468+0000","last_peered":"2024-04-23T18:19:41.690468+0000","last_clean":"2024-04-23T18:19:41.690468+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:19:41.690468+0000","last_undegraded":"2024-04-23T18:19:41.690468+0000","last_fullsized":"2024-04-23T18:19:41.690468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686815+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:19:41.686815+0000","last_peered":"2024-04-23T18:19:41.686815+0000","last_clean":"2024-04-23T18:19:41.686815+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:19:41.686815+0000","last_undegraded":"2024-04-23T18:19:41.686815+0000","last_fullsized":"2024-04-23T18:19:41.686815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.698372+0000","last_change":"2024-04-23T18:19:41.698372+0000","last_active":"2024-04-23T18:19:41.698372+0000","last_peered":"2024-04-23T18:19:41.698372+0000","last_clean":"2024-04-23T18:19:41.698372+0000","last_became_active":"2024-04-23T18:19:41.695014+0000","last_became_peered":"2024-04-23T18:19:41.695014+0000","last_unstale":"2024-04-23T18:19:41.698372+0000","last_undegraded":"2024-04-23T18:19:41.698372+0000","last_fullsized":"2024-04-23T18:19:41.698372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:35:15.568241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686170+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:19:41.686170+0000","last_peered":"2024-04-23T18:19:41.686170+0000","last_clean":"2024-04-23T18:19:41.686170+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:19:41.686170+0000","last_undegraded":"2024-04-23T18:19:41.686170+0000","last_fullsized":"2024-04-23T18:19:41.686170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690180+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:19:41.690180+0000","last_peered":"2024-04-23T18:19:41.690180+0000","last_clean":"2024-04-23T18:19:41.690180+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:19:41.690180+0000","last_undegraded":"2024-04-23T18:19:41.690180+0000","last_fullsized":"2024-04-23T18:19:41.690180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689254+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:19:41.689254+0000","last_peered":"2024-04-23T18:19:41.689254+0000","last_clean":"2024-04-23T18:19:41.689254+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:19:41.689254+0000","last_undegraded":"2024-04-23T18:19:41.689254+0000","last_fullsized":"2024-04-23T18:19:41.689254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690786+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:19:41.690786+0000","last_peered":"2024-04-23T18:19:41.690786+0000","last_clean":"2024-04-23T18:19:41.690786+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:19:41.690786+0000","last_undegraded":"2024-04-23T18:19:41.690786+0000","last_fullsized":"2024-04-23T18:19:41.690786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690516+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:19:41.690516+0000","last_peered":"2024-04-23T18:19:41.690516+0000","last_clean":"2024-04-23T18:19:41.690516+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:19:41.690516+0000","last_undegraded":"2024-04-23T18:19:41.690516+0000","last_fullsized":"2024-04-23T18:19:41.690516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002210+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:19:42.002210+0000","last_peered":"2024-04-23T18:19:42.002210+0000","last_clean":"2024-04-23T18:19:42.002210+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:19:42.002210+0000","last_undegraded":"2024-04-23T18:19:42.002210+0000","last_fullsized":"2024-04-23T18:19:42.002210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686213+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:19:41.686213+0000","last_peered":"2024-04-23T18:19:41.686213+0000","last_clean":"2024-04-23T18:19:41.686213+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:19:41.686213+0000","last_undegraded":"2024-04-23T18:19:41.686213+0000","last_fullsized":"2024-04-23T18:19:41.686213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689457+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:19:41.689457+0000","last_peered":"2024-04-23T18:19:41.689457+0000","last_clean":"2024-04-23T18:19:41.689457+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:19:41.689457+0000","last_undegraded":"2024-04-23T18:19:41.689457+0000","last_fullsized":"2024-04-23T18:19:41.689457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.691287+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:19:41.691287+0000","last_peered":"2024-04-23T18:19:41.691287+0000","last_clean":"2024-04-23T18:19:41.691287+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:19:41.691287+0000","last_undegraded":"2024-04-23T18:19:41.691287+0000","last_fullsized":"2024-04-23T18:19:41.691287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688573+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:19:41.688573+0000","last_peered":"2024-04-23T18:19:41.688573+0000","last_clean":"2024-04-23T18:19:41.688573+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:19:41.688573+0000","last_undegraded":"2024-04-23T18:19:41.688573+0000","last_fullsized":"2024-04-23T18:19:41.688573+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686326+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:19:41.686326+0000","last_peered":"2024-04-23T18:19:41.686326+0000","last_clean":"2024-04-23T18:19:41.686326+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:19:41.686326+0000","last_undegraded":"2024-04-23T18:19:41.686326+0000","last_fullsized":"2024-04-23T18:19:41.686326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688200+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:19:41.688200+0000","last_peered":"2024-04-23T18:19:41.688200+0000","last_clean":"2024-04-23T18:19:41.688200+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:19:41.688200+0000","last_undegraded":"2024-04-23T18:19:41.688200+0000","last_fullsized":"2024-04-23T18:19:41.688200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686346+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:19:41.686346+0000","last_peered":"2024-04-23T18:19:41.686346+0000","last_clean":"2024-04-23T18:19:41.686346+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:19:41.686346+0000","last_undegraded":"2024-04-23T18:19:41.686346+0000","last_fullsized":"2024-04-23T18:19:41.686346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686262+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:19:41.686262+0000","last_peered":"2024-04-23T18:19:41.686262+0000","last_clean":"2024-04-23T18:19:41.686262+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:19:41.686262+0000","last_undegraded":"2024-04-23T18:19:41.686262+0000","last_fullsized":"2024-04-23T18:19:41.686262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689285+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:19:41.689285+0000","last_peered":"2024-04-23T18:19:41.689285+0000","last_clean":"2024-04-23T18:19:41.689285+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:19:41.689285+0000","last_undegraded":"2024-04-23T18:19:41.689285+0000","last_fullsized":"2024-04-23T18:19:41.689285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":411,"num_read_kb":274,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":61080,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285595,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":472865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547233,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712324,"statfs":{"total":95995035648,"available":95961419776,"internally_reserved":0,"allocated":1683456,"data_stored":1125749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":473184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005098,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":473104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266735,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1597440,"data_stored":1065308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":472456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":533295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":473956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":472650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":534330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1605632,"data_stored":1064484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:19:49.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:19:49.982 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:50.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:50 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:50 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:50 smithi172 ceph-mon[32456]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:19:50.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:50 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:50.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:50 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:50.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:50 smithi172 ceph-mon[32456]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:50 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:50 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:50 smithi037 ceph-mon[29404]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:19:50.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:50 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:50.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:50 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:50.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:50 smithi037 ceph-mon[29404]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:50 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:50 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:50 smithi173 ceph-mon[32906]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:19:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:50 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:50 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:50 smithi173 ceph-mon[32906]: from='mgr.14150 172.21.15.37:0/505795075' entity='mgr.x' 2024-04-23T18:19:51.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:51 smithi172 ceph-mon[32456]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s; 6.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:51.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:51 smithi173 ceph-mon[32906]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s; 6.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:51.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:51 smithi037 ceph-mon[29404]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4 op/s; 6.5 KiB/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 -- 172.21.15.37:0/1242647203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec107c10 msgr2=0x7f6fec107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/1242647203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec107c10 0x7f6fec107ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80099c0 tx=0x7f6fe8009cd0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 -- 172.21.15.37:0/1242647203 shutdown_connections 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/1242647203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec107c10 0x7f6fec107ff0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/1242647203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fec102eb0 0x7f6fec10e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/1242647203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec1024f0 0x7f6fec102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.481+0000 7f6ff3a22700 1 -- 172.21.15.37:0/1242647203 >> 172.21.15.37:0/1242647203 conn(0x7f6fec076c60 msgr2=0x7f6fec077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 -- 172.21.15.37:0/1242647203 shutdown_connections 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 -- 172.21.15.37:0/1242647203 wait complete. 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 Processor -- start 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 -- start start 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 0x7f6fec19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fec102eb0 0x7f6fec19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec107c10 0x7f6fec1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec111cb0 con 0x7f6fec1024f0 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec111b30 con 0x7f6fec107c10 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.482+0000 7f6ff3a22700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec111e30 con 0x7f6fec102eb0 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 0x7f6fec19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 0x7f6fec19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50482/0 (socket says 172.21.15.37:50482) 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 -- 172.21.15.37:0/3476985093 learned_addr learned my addr 172.21.15.37:0/3476985093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:51.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff0fbd700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fec102eb0 0x7f6fec19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff1fbf700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec107c10 0x7f6fec1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 -- 172.21.15.37:0/3476985093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fec102eb0 msgr2=0x7f6fec19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fec102eb0 0x7f6fec19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 -- 172.21.15.37:0/3476985093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec107c10 msgr2=0x7f6fec1a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec107c10 0x7f6fec1a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 -- 172.21.15.37:0/3476985093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec1a5670 con 0x7f6fec1024f0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff1fbf700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec107c10 0x7f6fec1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff17be700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 0x7f6fec19a0b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800b580 tx=0x7f6fe80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8027070 con 0x7f6fec1024f0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec1a5840 con 0x7f6fec1024f0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe8003f20 con 0x7f6fec1024f0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8022d70 con 0x7f6fec1024f0 2024-04-23T18:19:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.483+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1a5c90 con 0x7f6fec1024f0 2024-04-23T18:19:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.484+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec069730 con 0x7f6fec1024f0 2024-04-23T18:19:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.486+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6fe8022500 con 0x7f6fec1024f0 2024-04-23T18:19:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.486+0000 7f6fe27fc700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6fd80a3cd0 0x7f6fd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.487+0000 7f6ff0fbd700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6fd80a3cd0 0x7f6fd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.487+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6fe80d1240 con 0x7f6fec1024f0 2024-04-23T18:19:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.487+0000 7f6ff0fbd700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6fd80a3cd0 0x7f6fd80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0772d0 tx=0x7f6fdc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:51.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.491+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fe8013070 con 0x7f6fec1024f0 2024-04-23T18:19:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.694+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fec062e80 con 0x7f6fd80a3cd0 2024-04-23T18:19:51.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.702+0000 7f6fe27fc700 1 -- 172.21.15.37:0/3476985093 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258729 (secure 0 0 0) 0x7f6fec062e80 con 0x7f6fd80a3cd0 2024-04-23T18:19:51.702 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6fd80a3cd0 msgr2=0x7f6fd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6fd80a3cd0 0x7f6fd80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0772d0 tx=0x7f6fdc009190 comp rx=0 tx=0).stop 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 msgr2=0x7f6fec19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 0x7f6fec19a0b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800b580 tx=0x7f6fe80047b0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 shutdown_connections 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6fd80a3cd0 0x7f6fd80a6190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fec1024f0 0x7f6fec19a0b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec107c10 0x7f6fec1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 --2- 172.21.15.37:0/3476985093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fec102eb0 0x7f6fec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 >> 172.21.15.37:0/3476985093 conn(0x7f6fec076c60 msgr2=0x7f6fec0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 shutdown_connections 2024-04-23T18:19:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:51.705+0000 7f6ff3a22700 1 -- 172.21.15.37:0/3476985093 wait complete. 2024-04-23T18:19:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:19:52.336 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-04-23T18:19:49.878532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":567,"num_read_kb":408,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1241933,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375412,"kb_used_data":13692,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566412,"statfs":{"total":1151940427776,"available":1151556005888,"internally_reserved":0,"allocated":14020608,"data_stored":7635108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":26,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61197,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.199366"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689574+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:19:41.689574+0000","last_peered":"2024-04-23T18:19:41.689574+0000","last_clean":"2024-04-23T18:19:41.689574+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:19:41.689574+0000","last_undegraded":"2024-04-23T18:19:41.689574+0000","last_fullsized":"2024-04-23T18:19:41.689574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002051+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:19:42.002051+0000","last_peered":"2024-04-23T18:19:42.002051+0000","last_clean":"2024-04-23T18:19:42.002051+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:19:42.002051+0000","last_undegraded":"2024-04-23T18:19:42.002051+0000","last_fullsized":"2024-04-23T18:19:42.002051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002082+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:19:42.002082+0000","last_peered":"2024-04-23T18:19:42.002082+0000","last_clean":"2024-04-23T18:19:42.002082+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:19:42.002082+0000","last_undegraded":"2024-04-23T18:19:42.002082+0000","last_fullsized":"2024-04-23T18:19:42.002082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.695047+0000","last_change":"2024-04-23T18:19:41.694639+0000","last_active":"2024-04-23T18:19:41.695047+0000","last_peered":"2024-04-23T18:19:41.695047+0000","last_clean":"2024-04-23T18:19:41.695047+0000","last_became_active":"2024-04-23T18:19:41.694291+0000","last_became_peered":"2024-04-23T18:19:41.694291+0000","last_unstale":"2024-04-23T18:19:41.695047+0000","last_undegraded":"2024-04-23T18:19:41.695047+0000","last_fullsized":"2024-04-23T18:19:41.695047+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:44.268737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692247+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:19:41.692247+0000","last_peered":"2024-04-23T18:19:41.692247+0000","last_clean":"2024-04-23T18:19:41.692247+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:19:41.692247+0000","last_undegraded":"2024-04-23T18:19:41.692247+0000","last_fullsized":"2024-04-23T18:19:41.692247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689447+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:19:41.689447+0000","last_peered":"2024-04-23T18:19:41.689447+0000","last_clean":"2024-04-23T18:19:41.689447+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:19:41.689447+0000","last_undegraded":"2024-04-23T18:19:41.689447+0000","last_fullsized":"2024-04-23T18:19:41.689447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689135+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:19:41.689135+0000","last_peered":"2024-04-23T18:19:41.689135+0000","last_clean":"2024-04-23T18:19:41.689135+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:19:41.689135+0000","last_undegraded":"2024-04-23T18:19:41.689135+0000","last_fullsized":"2024-04-23T18:19:41.689135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687857+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:19:41.687857+0000","last_peered":"2024-04-23T18:19:41.687857+0000","last_clean":"2024-04-23T18:19:41.687857+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:19:41.687857+0000","last_undegraded":"2024-04-23T18:19:41.687857+0000","last_fullsized":"2024-04-23T18:19:41.687857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687228+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:19:41.687228+0000","last_peered":"2024-04-23T18:19:41.687228+0000","last_clean":"2024-04-23T18:19:41.687228+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:19:41.687228+0000","last_undegraded":"2024-04-23T18:19:41.687228+0000","last_fullsized":"2024-04-23T18:19:41.687228+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688707+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:19:41.688707+0000","last_peered":"2024-04-23T18:19:41.688707+0000","last_clean":"2024-04-23T18:19:41.688707+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:19:41.688707+0000","last_undegraded":"2024-04-23T18:19:41.688707+0000","last_fullsized":"2024-04-23T18:19:41.688707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690352+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:19:41.690352+0000","last_peered":"2024-04-23T18:19:41.690352+0000","last_clean":"2024-04-23T18:19:41.690352+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:19:41.690352+0000","last_undegraded":"2024-04-23T18:19:41.690352+0000","last_fullsized":"2024-04-23T18:19:41.690352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.930064+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:19:41.930064+0000","last_peered":"2024-04-23T18:19:41.930064+0000","last_clean":"2024-04-23T18:19:41.930064+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:19:41.930064+0000","last_undegraded":"2024-04-23T18:19:41.930064+0000","last_fullsized":"2024-04-23T18:19:41.930064+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686269+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:19:41.686269+0000","last_peered":"2024-04-23T18:19:41.686269+0000","last_clean":"2024-04-23T18:19:41.686269+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:19:41.686269+0000","last_undegraded":"2024-04-23T18:19:41.686269+0000","last_fullsized":"2024-04-23T18:19:41.686269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.001902+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:19:42.001902+0000","last_peered":"2024-04-23T18:19:42.001902+0000","last_clean":"2024-04-23T18:19:42.001902+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:19:42.001902+0000","last_undegraded":"2024-04-23T18:19:42.001902+0000","last_fullsized":"2024-04-23T18:19:42.001902+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.001899+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:19:42.001899+0000","last_peered":"2024-04-23T18:19:42.001899+0000","last_clean":"2024-04-23T18:19:42.001899+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:19:42.001899+0000","last_undegraded":"2024-04-23T18:19:42.001899+0000","last_fullsized":"2024-04-23T18:19:42.001899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686059+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:19:41.686059+0000","last_peered":"2024-04-23T18:19:41.686059+0000","last_clean":"2024-04-23T18:19:41.686059+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:19:41.686059+0000","last_undegraded":"2024-04-23T18:19:41.686059+0000","last_fullsized":"2024-04-23T18:19:41.686059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689690+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:19:41.689690+0000","last_peered":"2024-04-23T18:19:41.689690+0000","last_clean":"2024-04-23T18:19:41.689690+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:19:41.689690+0000","last_undegraded":"2024-04-23T18:19:41.689690+0000","last_fullsized":"2024-04-23T18:19:41.689690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688150+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:19:41.688150+0000","last_peered":"2024-04-23T18:19:41.688150+0000","last_clean":"2024-04-23T18:19:41.688150+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:19:41.688150+0000","last_undegraded":"2024-04-23T18:19:41.688150+0000","last_fullsized":"2024-04-23T18:19:41.688150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690857+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:19:41.690857+0000","last_peered":"2024-04-23T18:19:41.690857+0000","last_clean":"2024-04-23T18:19:41.690857+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:19:41.690857+0000","last_undegraded":"2024-04-23T18:19:41.690857+0000","last_fullsized":"2024-04-23T18:19:41.690857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688368+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:19:41.688368+0000","last_peered":"2024-04-23T18:19:41.688368+0000","last_clean":"2024-04-23T18:19:41.688368+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:19:41.688368+0000","last_undegraded":"2024-04-23T18:19:41.688368+0000","last_fullsized":"2024-04-23T18:19:41.688368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688217+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:19:41.688217+0000","last_peered":"2024-04-23T18:19:41.688217+0000","last_clean":"2024-04-23T18:19:41.688217+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:19:41.688217+0000","last_undegraded":"2024-04-23T18:19:41.688217+0000","last_fullsized":"2024-04-23T18:19:41.688217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688479+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:19:41.688479+0000","last_peered":"2024-04-23T18:19:41.688479+0000","last_clean":"2024-04-23T18:19:41.688479+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:19:41.688479+0000","last_undegraded":"2024-04-23T18:19:41.688479+0000","last_fullsized":"2024-04-23T18:19:41.688479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002174+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:19:42.002174+0000","last_peered":"2024-04-23T18:19:42.002174+0000","last_clean":"2024-04-23T18:19:42.002174+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:19:42.002174+0000","last_undegraded":"2024-04-23T18:19:42.002174+0000","last_fullsized":"2024-04-23T18:19:42.002174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692597+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:19:41.692597+0000","last_peered":"2024-04-23T18:19:41.692597+0000","last_clean":"2024-04-23T18:19:41.692597+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:19:41.692597+0000","last_undegraded":"2024-04-23T18:19:41.692597+0000","last_fullsized":"2024-04-23T18:19:41.692597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686361+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:19:41.686361+0000","last_peered":"2024-04-23T18:19:41.686361+0000","last_clean":"2024-04-23T18:19:41.686361+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:19:41.686361+0000","last_undegraded":"2024-04-23T18:19:41.686361+0000","last_fullsized":"2024-04-23T18:19:41.686361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.693677+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:19:41.693677+0000","last_peered":"2024-04-23T18:19:41.693677+0000","last_clean":"2024-04-23T18:19:41.693677+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:19:41.693677+0000","last_undegraded":"2024-04-23T18:19:41.693677+0000","last_fullsized":"2024-04-23T18:19:41.693677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686834+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:19:41.686834+0000","last_peered":"2024-04-23T18:19:41.686834+0000","last_clean":"2024-04-23T18:19:41.686834+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:19:41.686834+0000","last_undegraded":"2024-04-23T18:19:41.686834+0000","last_fullsized":"2024-04-23T18:19:41.686834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:43.191144+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:19:43.191144+0000","last_peered":"2024-04-23T18:19:43.191144+0000","last_clean":"2024-04-23T18:19:43.191144+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:19:43.191144+0000","last_undegraded":"2024-04-23T18:19:43.191144+0000","last_fullsized":"2024-04-23T18:19:43.191144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692687+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:19:41.692687+0000","last_peered":"2024-04-23T18:19:41.692687+0000","last_clean":"2024-04-23T18:19:41.692687+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:19:41.692687+0000","last_undegraded":"2024-04-23T18:19:41.692687+0000","last_fullsized":"2024-04-23T18:19:41.692687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.933594+0000","last_change":"2024-04-23T18:19:41.933594+0000","last_active":"2024-04-23T18:19:41.933594+0000","last_peered":"2024-04-23T18:19:41.933594+0000","last_clean":"2024-04-23T18:19:41.933594+0000","last_became_active":"2024-04-23T18:19:41.930226+0000","last_became_peered":"2024-04-23T18:19:41.930226+0000","last_unstale":"2024-04-23T18:19:41.933594+0000","last_undegraded":"2024-04-23T18:19:41.933594+0000","last_fullsized":"2024-04-23T18:19:41.933594+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:57.183719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.694345+0000","last_change":"2024-04-23T18:19:41.694345+0000","last_active":"2024-04-23T18:19:41.694345+0000","last_peered":"2024-04-23T18:19:41.694345+0000","last_clean":"2024-04-23T18:19:41.694345+0000","last_became_active":"2024-04-23T18:19:41.693912+0000","last_became_peered":"2024-04-23T18:19:41.693912+0000","last_unstale":"2024-04-23T18:19:41.694345+0000","last_undegraded":"2024-04-23T18:19:41.694345+0000","last_fullsized":"2024-04-23T18:19:41.694345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:12:07.153403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686049+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:19:41.686049+0000","last_peered":"2024-04-23T18:19:41.686049+0000","last_clean":"2024-04-23T18:19:41.686049+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:19:41.686049+0000","last_undegraded":"2024-04-23T18:19:41.686049+0000","last_fullsized":"2024-04-23T18:19:41.686049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687916+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:19:41.687916+0000","last_peered":"2024-04-23T18:19:41.687916+0000","last_clean":"2024-04-23T18:19:41.687916+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:19:41.687916+0000","last_undegraded":"2024-04-23T18:19:41.687916+0000","last_fullsized":"2024-04-23T18:19:41.687916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686238+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:19:41.686238+0000","last_peered":"2024-04-23T18:19:41.686238+0000","last_clean":"2024-04-23T18:19:41.686238+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:19:41.686238+0000","last_undegraded":"2024-04-23T18:19:41.686238+0000","last_fullsized":"2024-04-23T18:19:41.686238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688522+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:19:41.688522+0000","last_peered":"2024-04-23T18:19:41.688522+0000","last_clean":"2024-04-23T18:19:41.688522+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:19:41.688522+0000","last_undegraded":"2024-04-23T18:19:41.688522+0000","last_fullsized":"2024-04-23T18:19:41.688522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:48.197052+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:19:48.197052+0000","last_peered":"2024-04-23T18:19:48.197052+0000","last_clean":"2024-04-23T18:19:48.197052+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:19:48.197052+0000","last_undegraded":"2024-04-23T18:19:48.197052+0000","last_fullsized":"2024-04-23T18:19:48.197052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.687560+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:19:41.687560+0000","last_peered":"2024-04-23T18:19:41.687560+0000","last_clean":"2024-04-23T18:19:41.687560+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:19:41.687560+0000","last_undegraded":"2024-04-23T18:19:41.687560+0000","last_fullsized":"2024-04-23T18:19:41.687560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690065+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:19:41.690065+0000","last_peered":"2024-04-23T18:19:41.690065+0000","last_clean":"2024-04-23T18:19:41.690065+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:19:41.690065+0000","last_undegraded":"2024-04-23T18:19:41.690065+0000","last_fullsized":"2024-04-23T18:19:41.690065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002134+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:19:42.002134+0000","last_peered":"2024-04-23T18:19:42.002134+0000","last_clean":"2024-04-23T18:19:42.002134+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:19:42.002134+0000","last_undegraded":"2024-04-23T18:19:42.002134+0000","last_fullsized":"2024-04-23T18:19:42.002134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686989+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:19:41.686989+0000","last_peered":"2024-04-23T18:19:41.686989+0000","last_clean":"2024-04-23T18:19:41.686989+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:19:41.686989+0000","last_undegraded":"2024-04-23T18:19:41.686989+0000","last_fullsized":"2024-04-23T18:19:41.686989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.694590+0000","last_change":"2024-04-23T18:19:41.694590+0000","last_active":"2024-04-23T18:19:41.694590+0000","last_peered":"2024-04-23T18:19:41.694590+0000","last_clean":"2024-04-23T18:19:41.694590+0000","last_became_active":"2024-04-23T18:19:41.694163+0000","last_became_peered":"2024-04-23T18:19:41.694163+0000","last_unstale":"2024-04-23T18:19:41.694590+0000","last_undegraded":"2024-04-23T18:19:41.694590+0000","last_fullsized":"2024-04-23T18:19:41.694590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:28.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'28","reported_seq":211,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:48.945054+0000","last_change":"2024-04-23T18:19:41.940140+0000","last_active":"2024-04-23T18:19:48.945054+0000","last_peered":"2024-04-23T18:19:48.945054+0000","last_clean":"2024-04-23T18:19:48.945054+0000","last_became_active":"2024-04-23T18:19:41.931221+0000","last_became_peered":"2024-04-23T18:19:41.931221+0000","last_unstale":"2024-04-23T18:19:48.945054+0000","last_undegraded":"2024-04-23T18:19:48.945054+0000","last_fullsized":"2024-04-23T18:19:48.945054+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:40.122956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":140,"num_read_kb":129,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692395+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:19:41.692395+0000","last_peered":"2024-04-23T18:19:41.692395+0000","last_clean":"2024-04-23T18:19:41.692395+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:19:41.692395+0000","last_undegraded":"2024-04-23T18:19:41.692395+0000","last_fullsized":"2024-04-23T18:19:41.692395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.700269+0000","last_change":"2024-04-23T18:19:41.699880+0000","last_active":"2024-04-23T18:19:41.700269+0000","last_peered":"2024-04-23T18:19:41.700269+0000","last_clean":"2024-04-23T18:19:41.700269+0000","last_became_active":"2024-04-23T18:19:41.691650+0000","last_became_peered":"2024-04-23T18:19:41.691650+0000","last_unstale":"2024-04-23T18:19:41.700269+0000","last_undegraded":"2024-04-23T18:19:41.700269+0000","last_fullsized":"2024-04-23T18:19:41.700269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:28.166844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.685994+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:19:41.685994+0000","last_peered":"2024-04-23T18:19:41.685994+0000","last_clean":"2024-04-23T18:19:41.685994+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:19:41.685994+0000","last_undegraded":"2024-04-23T18:19:41.685994+0000","last_fullsized":"2024-04-23T18:19:41.685994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686376+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:19:41.686376+0000","last_peered":"2024-04-23T18:19:41.686376+0000","last_clean":"2024-04-23T18:19:41.686376+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:19:41.686376+0000","last_undegraded":"2024-04-23T18:19:41.686376+0000","last_fullsized":"2024-04-23T18:19:41.686376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689253+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:19:41.689253+0000","last_peered":"2024-04-23T18:19:41.689253+0000","last_clean":"2024-04-23T18:19:41.689253+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:19:41.689253+0000","last_undegraded":"2024-04-23T18:19:41.689253+0000","last_fullsized":"2024-04-23T18:19:41.689253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:44.191870+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:19:44.191870+0000","last_peered":"2024-04-23T18:19:44.191870+0000","last_clean":"2024-04-23T18:19:44.191870+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:19:44.191870+0000","last_undegraded":"2024-04-23T18:19:44.191870+0000","last_fullsized":"2024-04-23T18:19:44.191870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688720+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:19:41.688720+0000","last_peered":"2024-04-23T18:19:41.688720+0000","last_clean":"2024-04-23T18:19:41.688720+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:19:41.688720+0000","last_undegraded":"2024-04-23T18:19:41.688720+0000","last_fullsized":"2024-04-23T18:19:41.688720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692312+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:19:41.692312+0000","last_peered":"2024-04-23T18:19:41.692312+0000","last_clean":"2024-04-23T18:19:41.692312+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:19:41.692312+0000","last_undegraded":"2024-04-23T18:19:41.692312+0000","last_fullsized":"2024-04-23T18:19:41.692312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686403+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:19:41.686403+0000","last_peered":"2024-04-23T18:19:41.686403+0000","last_clean":"2024-04-23T18:19:41.686403+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:19:41.686403+0000","last_undegraded":"2024-04-23T18:19:41.686403+0000","last_fullsized":"2024-04-23T18:19:41.686403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692281+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:19:41.692281+0000","last_peered":"2024-04-23T18:19:41.692281+0000","last_clean":"2024-04-23T18:19:41.692281+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:19:41.692281+0000","last_undegraded":"2024-04-23T18:19:41.692281+0000","last_fullsized":"2024-04-23T18:19:41.692281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689429+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:19:41.689429+0000","last_peered":"2024-04-23T18:19:41.689429+0000","last_clean":"2024-04-23T18:19:41.689429+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:19:41.689429+0000","last_undegraded":"2024-04-23T18:19:41.689429+0000","last_fullsized":"2024-04-23T18:19:41.689429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688511+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:19:41.688511+0000","last_peered":"2024-04-23T18:19:41.688511+0000","last_clean":"2024-04-23T18:19:41.688511+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:19:41.688511+0000","last_undegraded":"2024-04-23T18:19:41.688511+0000","last_fullsized":"2024-04-23T18:19:41.688511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688442+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:19:41.688442+0000","last_peered":"2024-04-23T18:19:41.688442+0000","last_clean":"2024-04-23T18:19:41.688442+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:19:41.688442+0000","last_undegraded":"2024-04-23T18:19:41.688442+0000","last_fullsized":"2024-04-23T18:19:41.688442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689502+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:19:41.689502+0000","last_peered":"2024-04-23T18:19:41.689502+0000","last_clean":"2024-04-23T18:19:41.689502+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:19:41.689502+0000","last_undegraded":"2024-04-23T18:19:41.689502+0000","last_fullsized":"2024-04-23T18:19:41.689502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.003042+0000","last_change":"2024-04-23T18:19:42.003042+0000","last_active":"2024-04-23T18:19:42.003042+0000","last_peered":"2024-04-23T18:19:42.003042+0000","last_clean":"2024-04-23T18:19:42.003042+0000","last_became_active":"2024-04-23T18:19:42.002728+0000","last_became_peered":"2024-04-23T18:19:42.002728+0000","last_unstale":"2024-04-23T18:19:42.003042+0000","last_undegraded":"2024-04-23T18:19:42.003042+0000","last_fullsized":"2024-04-23T18:19:42.003042+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:54:33.401918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686407+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:19:41.686407+0000","last_peered":"2024-04-23T18:19:41.686407+0000","last_clean":"2024-04-23T18:19:41.686407+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:19:41.686407+0000","last_undegraded":"2024-04-23T18:19:41.686407+0000","last_fullsized":"2024-04-23T18:19:41.686407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:48.196768+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:19:48.196768+0000","last_peered":"2024-04-23T18:19:48.196768+0000","last_clean":"2024-04-23T18:19:48.196768+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:19:48.196768+0000","last_undegraded":"2024-04-23T18:19:48.196768+0000","last_fullsized":"2024-04-23T18:19:48.196768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.691129+0000","last_change":"2024-04-23T18:19:41.691129+0000","last_active":"2024-04-23T18:19:41.691129+0000","last_peered":"2024-04-23T18:19:41.691129+0000","last_clean":"2024-04-23T18:19:41.691129+0000","last_became_active":"2024-04-23T18:19:41.690710+0000","last_became_peered":"2024-04-23T18:19:41.690710+0000","last_unstale":"2024-04-23T18:19:41.691129+0000","last_undegraded":"2024-04-23T18:19:41.691129+0000","last_fullsized":"2024-04-23T18:19:41.691129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:51.914794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.930135+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:19:41.930135+0000","last_peered":"2024-04-23T18:19:41.930135+0000","last_clean":"2024-04-23T18:19:41.930135+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:19:41.930135+0000","last_undegraded":"2024-04-23T18:19:41.930135+0000","last_fullsized":"2024-04-23T18:19:41.930135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689005+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:19:41.689005+0000","last_peered":"2024-04-23T18:19:41.689005+0000","last_clean":"2024-04-23T18:19:41.689005+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:19:41.689005+0000","last_undegraded":"2024-04-23T18:19:41.689005+0000","last_fullsized":"2024-04-23T18:19:41.689005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.685897+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:19:41.685897+0000","last_peered":"2024-04-23T18:19:41.685897+0000","last_clean":"2024-04-23T18:19:41.685897+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:19:41.685897+0000","last_undegraded":"2024-04-23T18:19:41.685897+0000","last_fullsized":"2024-04-23T18:19:41.685897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689084+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:19:41.689084+0000","last_peered":"2024-04-23T18:19:41.689084+0000","last_clean":"2024-04-23T18:19:41.689084+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:19:41.689084+0000","last_undegraded":"2024-04-23T18:19:41.689084+0000","last_fullsized":"2024-04-23T18:19:41.689084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692563+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:19:41.692563+0000","last_peered":"2024-04-23T18:19:41.692563+0000","last_clean":"2024-04-23T18:19:41.692563+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:19:41.692563+0000","last_undegraded":"2024-04-23T18:19:41.692563+0000","last_fullsized":"2024-04-23T18:19:41.692563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686127+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:19:41.686127+0000","last_peered":"2024-04-23T18:19:41.686127+0000","last_clean":"2024-04-23T18:19:41.686127+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:19:41.686127+0000","last_undegraded":"2024-04-23T18:19:41.686127+0000","last_fullsized":"2024-04-23T18:19:41.686127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688371+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:19:41.688371+0000","last_peered":"2024-04-23T18:19:41.688371+0000","last_clean":"2024-04-23T18:19:41.688371+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:19:41.688371+0000","last_undegraded":"2024-04-23T18:19:41.688371+0000","last_fullsized":"2024-04-23T18:19:41.688371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688615+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:19:41.688615+0000","last_peered":"2024-04-23T18:19:41.688615+0000","last_clean":"2024-04-23T18:19:41.688615+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:19:41.688615+0000","last_undegraded":"2024-04-23T18:19:41.688615+0000","last_fullsized":"2024-04-23T18:19:41.688615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.929678+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:19:41.929678+0000","last_peered":"2024-04-23T18:19:41.929678+0000","last_clean":"2024-04-23T18:19:41.929678+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:19:41.929678+0000","last_undegraded":"2024-04-23T18:19:41.929678+0000","last_fullsized":"2024-04-23T18:19:41.929678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002179+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:19:42.002179+0000","last_peered":"2024-04-23T18:19:42.002179+0000","last_clean":"2024-04-23T18:19:42.002179+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:19:42.002179+0000","last_undegraded":"2024-04-23T18:19:42.002179+0000","last_fullsized":"2024-04-23T18:19:42.002179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686081+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:19:41.686081+0000","last_peered":"2024-04-23T18:19:41.686081+0000","last_clean":"2024-04-23T18:19:41.686081+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:19:41.686081+0000","last_undegraded":"2024-04-23T18:19:41.686081+0000","last_fullsized":"2024-04-23T18:19:41.686081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692397+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:19:41.692397+0000","last_peered":"2024-04-23T18:19:41.692397+0000","last_clean":"2024-04-23T18:19:41.692397+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:19:41.692397+0000","last_undegraded":"2024-04-23T18:19:41.692397+0000","last_fullsized":"2024-04-23T18:19:41.692397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688264+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:19:41.688264+0000","last_peered":"2024-04-23T18:19:41.688264+0000","last_clean":"2024-04-23T18:19:41.688264+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:19:41.688264+0000","last_undegraded":"2024-04-23T18:19:41.688264+0000","last_fullsized":"2024-04-23T18:19:41.688264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689190+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:19:41.689190+0000","last_peered":"2024-04-23T18:19:41.689190+0000","last_clean":"2024-04-23T18:19:41.689190+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:19:41.689190+0000","last_undegraded":"2024-04-23T18:19:41.689190+0000","last_fullsized":"2024-04-23T18:19:41.689190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690975+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:19:41.690975+0000","last_peered":"2024-04-23T18:19:41.690975+0000","last_clean":"2024-04-23T18:19:41.690975+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:19:41.690975+0000","last_undegraded":"2024-04-23T18:19:41.690975+0000","last_fullsized":"2024-04-23T18:19:41.690975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688094+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:19:41.688094+0000","last_peered":"2024-04-23T18:19:41.688094+0000","last_clean":"2024-04-23T18:19:41.688094+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:19:41.688094+0000","last_undegraded":"2024-04-23T18:19:41.688094+0000","last_fullsized":"2024-04-23T18:19:41.688094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.692137+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:19:41.692137+0000","last_peered":"2024-04-23T18:19:41.692137+0000","last_clean":"2024-04-23T18:19:41.692137+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:19:41.692137+0000","last_undegraded":"2024-04-23T18:19:41.692137+0000","last_fullsized":"2024-04-23T18:19:41.692137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689640+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:19:41.689640+0000","last_peered":"2024-04-23T18:19:41.689640+0000","last_clean":"2024-04-23T18:19:41.689640+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:19:41.689640+0000","last_undegraded":"2024-04-23T18:19:41.689640+0000","last_fullsized":"2024-04-23T18:19:41.689640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690835+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:19:41.690835+0000","last_peered":"2024-04-23T18:19:41.690835+0000","last_clean":"2024-04-23T18:19:41.690835+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:19:41.690835+0000","last_undegraded":"2024-04-23T18:19:41.690835+0000","last_fullsized":"2024-04-23T18:19:41.690835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690468+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:19:41.690468+0000","last_peered":"2024-04-23T18:19:41.690468+0000","last_clean":"2024-04-23T18:19:41.690468+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:19:41.690468+0000","last_undegraded":"2024-04-23T18:19:41.690468+0000","last_fullsized":"2024-04-23T18:19:41.690468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686815+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:19:41.686815+0000","last_peered":"2024-04-23T18:19:41.686815+0000","last_clean":"2024-04-23T18:19:41.686815+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:19:41.686815+0000","last_undegraded":"2024-04-23T18:19:41.686815+0000","last_fullsized":"2024-04-23T18:19:41.686815+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.698372+0000","last_change":"2024-04-23T18:19:41.698372+0000","last_active":"2024-04-23T18:19:41.698372+0000","last_peered":"2024-04-23T18:19:41.698372+0000","last_clean":"2024-04-23T18:19:41.698372+0000","last_became_active":"2024-04-23T18:19:41.695014+0000","last_became_peered":"2024-04-23T18:19:41.695014+0000","last_unstale":"2024-04-23T18:19:41.698372+0000","last_undegraded":"2024-04-23T18:19:41.698372+0000","last_fullsized":"2024-04-23T18:19:41.698372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:35:15.568241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686170+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:19:41.686170+0000","last_peered":"2024-04-23T18:19:41.686170+0000","last_clean":"2024-04-23T18:19:41.686170+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:19:41.686170+0000","last_undegraded":"2024-04-23T18:19:41.686170+0000","last_fullsized":"2024-04-23T18:19:41.686170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690180+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:19:41.690180+0000","last_peered":"2024-04-23T18:19:41.690180+0000","last_clean":"2024-04-23T18:19:41.690180+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:19:41.690180+0000","last_undegraded":"2024-04-23T18:19:41.690180+0000","last_fullsized":"2024-04-23T18:19:41.690180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689254+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:19:41.689254+0000","last_peered":"2024-04-23T18:19:41.689254+0000","last_clean":"2024-04-23T18:19:41.689254+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:19:41.689254+0000","last_undegraded":"2024-04-23T18:19:41.689254+0000","last_fullsized":"2024-04-23T18:19:41.689254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690786+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:19:41.690786+0000","last_peered":"2024-04-23T18:19:41.690786+0000","last_clean":"2024-04-23T18:19:41.690786+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:19:41.690786+0000","last_undegraded":"2024-04-23T18:19:41.690786+0000","last_fullsized":"2024-04-23T18:19:41.690786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.690516+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:19:41.690516+0000","last_peered":"2024-04-23T18:19:41.690516+0000","last_clean":"2024-04-23T18:19:41.690516+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:19:41.690516+0000","last_undegraded":"2024-04-23T18:19:41.690516+0000","last_fullsized":"2024-04-23T18:19:41.690516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:42.002210+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:19:42.002210+0000","last_peered":"2024-04-23T18:19:42.002210+0000","last_clean":"2024-04-23T18:19:42.002210+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:19:42.002210+0000","last_undegraded":"2024-04-23T18:19:42.002210+0000","last_fullsized":"2024-04-23T18:19:42.002210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686213+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:19:41.686213+0000","last_peered":"2024-04-23T18:19:41.686213+0000","last_clean":"2024-04-23T18:19:41.686213+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:19:41.686213+0000","last_undegraded":"2024-04-23T18:19:41.686213+0000","last_fullsized":"2024-04-23T18:19:41.686213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689457+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:19:41.689457+0000","last_peered":"2024-04-23T18:19:41.689457+0000","last_clean":"2024-04-23T18:19:41.689457+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:19:41.689457+0000","last_undegraded":"2024-04-23T18:19:41.689457+0000","last_fullsized":"2024-04-23T18:19:41.689457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.691287+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:19:41.691287+0000","last_peered":"2024-04-23T18:19:41.691287+0000","last_clean":"2024-04-23T18:19:41.691287+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:19:41.691287+0000","last_undegraded":"2024-04-23T18:19:41.691287+0000","last_fullsized":"2024-04-23T18:19:41.691287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688573+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:19:41.688573+0000","last_peered":"2024-04-23T18:19:41.688573+0000","last_clean":"2024-04-23T18:19:41.688573+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:19:41.688573+0000","last_undegraded":"2024-04-23T18:19:41.688573+0000","last_fullsized":"2024-04-23T18:19:41.688573+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686326+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:19:41.686326+0000","last_peered":"2024-04-23T18:19:41.686326+0000","last_clean":"2024-04-23T18:19:41.686326+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:19:41.686326+0000","last_undegraded":"2024-04-23T18:19:41.686326+0000","last_fullsized":"2024-04-23T18:19:41.686326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.688200+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:19:41.688200+0000","last_peered":"2024-04-23T18:19:41.688200+0000","last_clean":"2024-04-23T18:19:41.688200+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:19:41.688200+0000","last_undegraded":"2024-04-23T18:19:41.688200+0000","last_fullsized":"2024-04-23T18:19:41.688200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686346+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:19:41.686346+0000","last_peered":"2024-04-23T18:19:41.686346+0000","last_clean":"2024-04-23T18:19:41.686346+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:19:41.686346+0000","last_undegraded":"2024-04-23T18:19:41.686346+0000","last_fullsized":"2024-04-23T18:19:41.686346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.686262+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:19:41.686262+0000","last_peered":"2024-04-23T18:19:41.686262+0000","last_clean":"2024-04-23T18:19:41.686262+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:19:41.686262+0000","last_undegraded":"2024-04-23T18:19:41.686262+0000","last_fullsized":"2024-04-23T18:19:41.686262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-23T18:19:41.689285+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:19:41.689285+0000","last_peered":"2024-04-23T18:19:41.689285+0000","last_clean":"2024-04-23T18:19:41.689285+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:19:41.689285+0000","last_undegraded":"2024-04-23T18:19:41.689285+0000","last_fullsized":"2024-04-23T18:19:41.689285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":431,"num_read_kb":289,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":61080,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285596,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":472865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547234,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712324,"statfs":{"total":95995035648,"available":95961419776,"internally_reserved":0,"allocated":1683456,"data_stored":1125749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":473184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005099,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":473104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266736,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1597440,"data_stored":1065308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":472456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":533295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":473956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":472650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":534330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1605632,"data_stored":1064484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:19:52.340 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T18:19:52.340 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T18:19:52.340 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T18:19:52.340 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T18:19:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:53.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:53 smithi172 ceph-mon[32456]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:19:53.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:53 smithi172 ceph-mon[32456]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 5.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:19:53.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:53 smithi173 ceph-mon[32906]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:19:53.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:53 smithi173 ceph-mon[32906]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 5.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:19:53.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:53 smithi037 ceph-mon[29404]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:19:53.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:53 smithi037 ceph-mon[29404]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 5.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/1757357744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0107990 msgr2=0x7f9ac0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/1757357744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0107990 0x7f9ac0107d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009990 tx=0x7f9ab4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/1757357744 shutdown_connections 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/1757357744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0107990 0x7f9ac0107d70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/1757357744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac0102300 0x7f9ac010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/1757357744 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 0x7f9ac0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/1757357744 >> 172.21.15.37:0/1757357744 conn(0x7f9ac00788f0 msgr2=0x7f9ac0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/1757357744 shutdown_connections 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.238+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/1757357744 wait complete. 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 Processor -- start 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 -- start start 2024-04-23T18:19:54.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 0x7f9ac0108800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 0x7f9ac0108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac0107990 0x7f9ac0109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0111bd0 con 0x7f9ac0102300 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0111a50 con 0x7f9ac0107990 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac6ad3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0111d50 con 0x7f9ac0101940 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9abffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 0x7f9ac0108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9abffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 0x7f9ac0108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50494/0 (socket says 172.21.15.37:50494) 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9abffff700 1 -- 172.21.15.37:0/2362749033 learned_addr learned my addr 172.21.15.37:0/2362749033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.239+0000 7f9ac486f700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 0x7f9ac0108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:54.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac5070700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac0107990 0x7f9ac0109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:54.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac486f700 1 -- 172.21.15.37:0/2362749033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac0107990 msgr2=0x7f9ac0109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac486f700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac0107990 0x7f9ac0109280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac486f700 1 -- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 msgr2=0x7f9ac0108d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac486f700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 0x7f9ac0108d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac486f700 1 -- 172.21.15.37:0/2362749033 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac01abaf0 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9abffff700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 0x7f9ac0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac486f700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 0x7f9ac0108800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4027070 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab4003f20 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac01abd80 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.240+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4022d20 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.241+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac01ac180 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.242+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac0103060 con 0x7f9ac0101940 2024-04-23T18:19:54.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.244+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ab402a070 con 0x7f9ac0101940 2024-04-23T18:19:54.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.244+0000 7f9abdffb700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3cd0 0x7f9ab00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:54.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.244+0000 7f9abffff700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3cd0 0x7f9ab00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:54.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.244+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9ab40d1640 con 0x7f9ac0101940 2024-04-23T18:19:54.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.245+0000 7f9abffff700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3cd0 0x7f9ab00a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0109fc0 tx=0x7f9aac009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:54.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.247+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ab4013070 con 0x7f9ac0101940 2024-04-23T18:19:54.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.504+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9ac01ac4f0 con 0x7f9ac0101940 2024-04-23T18:19:54.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.505+0000 7f9abdffb700 1 -- 172.21.15.37:0/2362749033 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9ab409df40 con 0x7f9ac0101940 2024-04-23T18:19:54.498 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:54.498 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T18:19:54.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3cd0 msgr2=0x7f9ab00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3cd0 0x7f9ab00a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0109fc0 tx=0x7f9aac009290 comp rx=0 tx=0).stop 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 msgr2=0x7f9ac0108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 0x7f9ac0108800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab40047b0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 shutdown_connections 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3cd0 0x7f9ab00a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac0107990 0x7f9ac0109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac0101940 0x7f9ac0108800 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 --2- 172.21.15.37:0/2362749033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac0102300 0x7f9ac0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 >> 172.21.15.37:0/2362749033 conn(0x7f9ac00788f0 msgr2=0x7f9ac006a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 shutdown_connections 2024-04-23T18:19:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:54.507+0000 7f9ac6ad3700 1 -- 172.21.15.37:0/2362749033 wait complete. 2024-04-23T18:19:55.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:55 smithi037 ceph-mon[29404]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:19:55.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:55 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2362749033' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:19:55.206 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T18:19:55.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:19:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 -- 172.21.15.37:0/528984568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c107c70 msgr2=0x7f519c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/528984568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c107c70 0x7f519c10a0c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5194009ca0 tx=0x7f519402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 -- 172.21.15.37:0/528984568 shutdown_connections 2024-04-23T18:19:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/528984568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c107c70 0x7f519c10a0c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/528984568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f519c105340 0x7f519c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/528984568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c102a10 0x7f519c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 -- 172.21.15.37:0/528984568 >> 172.21.15.37:0/528984568 conn(0x7f519c0fc400 msgr2=0x7f519c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 -- 172.21.15.37:0/528984568 shutdown_connections 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.319+0000 7f51a1ee1640 1 -- 172.21.15.37:0/528984568 wait complete. 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 Processor -- start 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 -- start start 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 0x7f519c102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c105340 0x7f519c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f519c107c70 0x7f519c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10dac0 con 0x7f519c102a10 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d940 con 0x7f519c105340 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f51a1ee1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10dc40 con 0x7f519c107c70 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 0x7f519c102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 0x7f519c102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58150/0 (socket says 172.21.15.37:58150) 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.320+0000 7f519b7fe640 1 -- 172.21.15.37:0/251193914 learned_addr learned my addr 172.21.15.37:0/251193914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519affd640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c105340 0x7f519c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519bfff640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f519c107c70 0x7f519c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519b7fe640 1 -- 172.21.15.37:0/251193914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f519c107c70 msgr2=0x7f519c100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519b7fe640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f519c107c70 0x7f519c100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519b7fe640 1 -- 172.21.15.37:0/251193914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c105340 msgr2=0x7f519c1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519b7fe640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c105340 0x7f519c1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519b7fe640 1 -- 172.21.15.37:0/251193914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c101790 con 0x7f519c102a10 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519affd640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c105340 0x7f519c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f519b7fe640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 0x7f519c102380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c920 tx=0x7f518c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c007c70 con 0x7f519c102a10 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c1a2220 con 0x7f519c102a10 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518c007e10 con 0x7f519c102a10 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c018820 con 0x7f519c102a10 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.321+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c1a2770 con 0x7f519c102a10 2024-04-23T18:19:55.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.322+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c0fdef0 con 0x7f519c102a10 2024-04-23T18:19:55.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.323+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f518c00f450 con 0x7f519c102a10 2024-04-23T18:19:55.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.324+0000 7f5198ff9640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f51840a3c40 0x7f51840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.324+0000 7f519affd640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f51840a3c40 0x7f51840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.324+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f518c0d0840 con 0x7f519c102a10 2024-04-23T18:19:55.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.324+0000 7f519affd640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f51840a3c40 0x7f51840a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51900058c0 tx=0x7f5190005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:55.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.332+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f518c014030 con 0x7f519c102a10 2024-04-23T18:19:55.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:55 smithi172 ceph-mon[32456]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:19:55.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:55 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2362749033' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:19:55.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:55 smithi173 ceph-mon[32906]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:19:55.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:55 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2362749033' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:19:55.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.521+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f519c062b50 con 0x7f519c102a10 2024-04-23T18:19:55.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.522+0000 7f5198ff9640 1 -- 172.21.15.37:0/251193914 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22684 (secure 0 0 0) 0x7f518c00de20 con 0x7f519c102a10 2024-04-23T18:19:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:55.516 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:19:41.678555+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:19:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f51840a3c40 msgr2=0x7f51840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f51840a3c40 0x7f51840a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51900058c0 tx=0x7f5190005850 comp rx=0 tx=0).stop 2024-04-23T18:19:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 msgr2=0x7f519c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 0x7f519c102380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c920 tx=0x7f518c00cde0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 shutdown_connections 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f51840a3c40 0x7f51840a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f519c107c70 0x7f519c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f519c105340 0x7f519c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 --2- 172.21.15.37:0/251193914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f519c102a10 0x7f519c102380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 >> 172.21.15.37:0/251193914 conn(0x7f519c0fc400 msgr2=0x7f519c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 shutdown_connections 2024-04-23T18:19:55.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.525+0000 7f51a1ee1640 1 -- 172.21.15.37:0/251193914 wait complete. 2024-04-23T18:19:55.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:19:55.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 -- 172.21.15.37:0/342672558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34105360 msgr2=0x7fea34107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 --2- 172.21.15.37:0/342672558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34105360 0x7fea34107750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009900 tx=0x7fea1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 -- 172.21.15.37:0/342672558 shutdown_connections 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 --2- 172.21.15.37:0/342672558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea34107c90 0x7fea3410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 --2- 172.21.15.37:0/342672558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34105360 0x7fea34107750 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 --2- 172.21.15.37:0/342672558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 0x7fea34104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 -- 172.21.15.37:0/342672558 >> 172.21.15.37:0/342672558 conn(0x7fea340fc440 msgr2=0x7fea340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 -- 172.21.15.37:0/342672558 shutdown_connections 2024-04-23T18:19:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 -- 172.21.15.37:0/342672558 wait complete. 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.643+0000 7fea39d9b640 1 Processor -- start 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 -- start start 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 0x7fea34198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea34105360 0x7fea341987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 0x7fea3419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410dab0 con 0x7fea34107c90 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410d930 con 0x7fea34105360 2024-04-23T18:19:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea39d9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410dc30 con 0x7fea34102a30 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 0x7fea3419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 0x7fea3419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58170/0 (socket says 172.21.15.37:58170) 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 -- 172.21.15.37:0/3853741692 learned_addr learned my addr 172.21.15.37:0/3853741692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea337fe640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 0x7fea34198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea32ffd640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea34105360 0x7fea341987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 -- 172.21.15.37:0/3853741692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 msgr2=0x7fea34198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 0x7fea34198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 -- 172.21.15.37:0/3853741692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea34105360 msgr2=0x7fea341987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea34105360 0x7fea341987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 -- 172.21.15.37:0/3853741692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3419fe90 con 0x7fea34107c90 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea337fe640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 0x7fea34198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea33fff640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 0x7fea3419f820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00cad0 tx=0x7fea2c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.644+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00ae20 con 0x7fea34107c90 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.645+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea341a0180 con 0x7fea34107c90 2024-04-23T18:19:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.645+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c005020 con 0x7fea34107c90 2024-04-23T18:19:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.645+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00f6b0 con 0x7fea34107c90 2024-04-23T18:19:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.645+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341a0680 con 0x7fea34107c90 2024-04-23T18:19:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.646+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea340fdf30 con 0x7fea34107c90 2024-04-23T18:19:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.647+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea2c0051d0 con 0x7fea34107c90 2024-04-23T18:19:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.647+0000 7fea30ff9640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea200a3c40 0x7fea200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.647+0000 7fea337fe640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea200a3c40 0x7fea200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.647+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fea2c0d0710 con 0x7fea34107c90 2024-04-23T18:19:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.648+0000 7fea337fe640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea200a3c40 0x7fea200a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea24009b40 tx=0x7fea24005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:55.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.652+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea2c014030 con 0x7fea34107c90 2024-04-23T18:19:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.837+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea34062b50 con 0x7fea34107c90 2024-04-23T18:19:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.838+0000 7fea30ff9640 1 -- 172.21.15.37:0/3853741692 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fea2c09cf80 con 0x7fea34107c90 2024-04-23T18:19:55.831 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:19:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea200a3c40 msgr2=0x7fea200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea200a3c40 0x7fea200a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea24009b40 tx=0x7fea24005e50 comp rx=0 tx=0).stop 2024-04-23T18:19:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 msgr2=0x7fea3419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 0x7fea3419f820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00cad0 tx=0x7fea2c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 shutdown_connections 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea200a3c40 0x7fea200a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea34107c90 0x7fea3419f820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea34105360 0x7fea341987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 --2- 172.21.15.37:0/3853741692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea34102a30 0x7fea34198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 >> 172.21.15.37:0/3853741692 conn(0x7fea340fc440 msgr2=0x7fea341034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 shutdown_connections 2024-04-23T18:19:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.840+0000 7fea39d9b640 1 -- 172.21.15.37:0/3853741692 wait complete. 2024-04-23T18:19:55.844 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.955+0000 7f7d59631640 1 -- 172.21.15.37:0/3352236400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 msgr2=0x7f7d5410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 --2- 172.21.15.37:0/3352236400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5410a3f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009ca0 tx=0x7f7d4c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 -- 172.21.15.37:0/3352236400 shutdown_connections 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 --2- 172.21.15.37:0/3352236400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5410a3f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 --2- 172.21.15.37:0/3352236400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 0x7f7d54103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 --2- 172.21.15.37:0/3352236400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101fb0 0x7f7d541023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 -- 172.21.15.37:0/3352236400 >> 172.21.15.37:0/3352236400 conn(0x7f7d540fd540 msgr2=0x7f7d540ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:55.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 -- 172.21.15.37:0/3352236400 shutdown_connections 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 -- 172.21.15.37:0/3352236400 wait complete. 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 Processor -- start 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 -- start start 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101fb0 0x7f7d54198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.956+0000 7f7d59631640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 0x7f7d54198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d59631640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d59631640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410daf0 con 0x7f7d54103b60 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d59631640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d970 con 0x7f7d54101fb0 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d59631640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410dc70 con 0x7f7d541031a0 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58192/0 (socket says 172.21.15.37:58192) 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 -- 172.21.15.37:0/13385555 learned_addr learned my addr 172.21.15.37:0/13385555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:55.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d527fc640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 0x7f7d54198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 -- 172.21.15.37:0/13385555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 msgr2=0x7f7d54198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 0x7f7d54198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 -- 172.21.15.37:0/13385555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101fb0 msgr2=0x7f7d54198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101fb0 0x7f7d54198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 -- 172.21.15.37:0/13385555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5419fd70 con 0x7f7d54103b60 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d527fc640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 0x7f7d54198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d537fe640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5419f700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0097b0 tx=0x7f7d4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004240 con 0x7f7d54103b60 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4c0317c0 con 0x7f7d54103b60 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d541a0000 con 0x7f7d54103b60 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c02fd80 con 0x7f7d54103b60 2024-04-23T18:19:55.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.957+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d541a0550 con 0x7f7d54103b60 2024-04-23T18:19:55.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.958+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d54109350 con 0x7f7d54103b60 2024-04-23T18:19:55.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.959+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d4c0043e0 con 0x7f7d54103b60 2024-04-23T18:19:55.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.960+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7d3c0a3c40 0x7f7d3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:55.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.960+0000 7f7d52ffd640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7d3c0a3c40 0x7f7d3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:55.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.960+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f7d4c0f1df0 con 0x7f7d54103b60 2024-04-23T18:19:55.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.960+0000 7f7d52ffd640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7d3c0a3c40 0x7f7d3c0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d44005fd0 tx=0x7f7d4400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:55.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:55.964+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d4c0be710 con 0x7f7d54103b60 2024-04-23T18:19:56.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:56 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/251193914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:56.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:56 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3853741692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:19:56.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.150+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d5402d7b0 con 0x7f7d54103b60 2024-04-23T18:19:56.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.151+0000 7f7d3bfff640 1 -- 172.21.15.37:0/13385555 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f7d4c0098b0 con 0x7f7d54103b60 2024-04-23T18:19:56.144 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7d3c0a3c40 msgr2=0x7f7d3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7d3c0a3c40 0x7f7d3c0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d44005fd0 tx=0x7f7d4400b410 comp rx=0 tx=0).stop 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 msgr2=0x7f7d5419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5419f700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0097b0 tx=0x7f7d4c003c20 comp rx=0 tx=0).stop 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 shutdown_connections 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7d3c0a3c40 0x7f7d3c0a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d54103b60 0x7f7d5419f700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d541031a0 0x7f7d54198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 --2- 172.21.15.37:0/13385555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101fb0 0x7f7d54198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 >> 172.21.15.37:0/13385555 conn(0x7f7d540fd540 msgr2=0x7f7d54106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:56.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 shutdown_connections 2024-04-23T18:19:56.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.153+0000 7f7d59631640 1 -- 172.21.15.37:0/13385555 wait complete. 2024-04-23T18:19:56.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:19:56.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/3986500482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 msgr2=0x7f7f4810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/3986500482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4810a0e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009ca0 tx=0x7f7f4002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/3986500482 shutdown_connections 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/3986500482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4810a0e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/3986500482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f48105360 0x7f7f48107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/3986500482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f48102a30 0x7f7f48104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/3986500482 >> 172.21.15.37:0/3986500482 conn(0x7f7f480fc440 msgr2=0x7f7f480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/3986500482 shutdown_connections 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.268+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/3986500482 wait complete. 2024-04-23T18:19:56.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 Processor -- start 2024-04-23T18:19:56.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 -- start start 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f48102a30 0x7f7f481981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f48105360 0x7f7f48198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810daf0 con 0x7f7f48107c90 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810d970 con 0x7f7f48105360 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4e7e2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810dc70 con 0x7f7f48102a30 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58200/0 (socket says 172.21.15.37:58200) 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 -- 172.21.15.37:0/139589428 learned_addr learned my addr 172.21.15.37:0/139589428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:56.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f47fff640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f48102a30 0x7f7f481981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f477fe640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f48105360 0x7f7f48198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 -- 172.21.15.37:0/139589428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f48102a30 msgr2=0x7f7f481981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f48102a30 0x7f7f481981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 -- 172.21.15.37:0/139589428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f48105360 msgr2=0x7f7f48198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f48105360 0x7f7f48198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.269+0000 7f7f4cd7f640 1 -- 172.21.15.37:0/139589428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4819fde0 con 0x7f7f48107c90 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.270+0000 7f7f4cd7f640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4819f770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f400097b0 tx=0x7f7f40003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.270+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40004240 con 0x7f7f48107c90 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.270+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f400317c0 con 0x7f7f48107c90 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.270+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f481a0070 con 0x7f7f48107c90 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.270+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4002fd80 con 0x7f7f48107c90 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.270+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481a05c0 con 0x7f7f48107c90 2024-04-23T18:19:56.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.271+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f480fdf30 con 0x7f7f48107c90 2024-04-23T18:19:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.272+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f400043e0 con 0x7f7f48107c90 2024-04-23T18:19:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.272+0000 7f7f457fa640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7f300a3c40 0x7f7f300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.273+0000 7f7f47fff640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7f300a3c40 0x7f7f300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.273+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f7f400f1d20 con 0x7f7f48107c90 2024-04-23T18:19:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.273+0000 7f7f47fff640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7f300a3c40 0x7f7f300a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00bbe0 tx=0x7f7f3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.277+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f400be6a0 con 0x7f7f48107c90 2024-04-23T18:19:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:56 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/251193914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:56.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:56 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3853741692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:19:56.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:56 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/251193914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:56.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:56 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3853741692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:19:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.463+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f48062b50 con 0x7f7f48107c90 2024-04-23T18:19:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.463+0000 7f7f457fa640 1 -- 172.21.15.37:0/139589428 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f7f400098b0 con 0x7f7f48107c90 2024-04-23T18:19:56.457 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7f300a3c40 msgr2=0x7f7f300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7f300a3c40 0x7f7f300a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00bbe0 tx=0x7f7f3c00b390 comp rx=0 tx=0).stop 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 msgr2=0x7f7f4819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4819f770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f400097b0 tx=0x7f7f40003c20 comp rx=0 tx=0).stop 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 shutdown_connections 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7f300a3c40 0x7f7f300a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f48107c90 0x7f7f4819f770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f48105360 0x7f7f48198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 --2- 172.21.15.37:0/139589428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f48102a30 0x7f7f481981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.465+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 >> 172.21.15.37:0/139589428 conn(0x7f7f480fc440 msgr2=0x7f7f48103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:56.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.466+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 shutdown_connections 2024-04-23T18:19:56.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.466+0000 7f7f4e7e2640 1 -- 172.21.15.37:0/139589428 wait complete. 2024-04-23T18:19:56.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:19:56.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 -- 172.21.15.37:0/2170010522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c101fe0 msgr2=0x7ff91c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2170010522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c101fe0 0x7ff91c102460 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff910009900 tx=0x7ff91002a410 comp rx=0 tx=0).stop 2024-04-23T18:19:56.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 -- 172.21.15.37:0/2170010522 shutdown_connections 2024-04-23T18:19:56.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2170010522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91c1029a0 0x7ff91c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2170010522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c101fe0 0x7ff91c102460 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2170010522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91c100e90 0x7ff91c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.578+0000 7ff9236ae640 1 -- 172.21.15.37:0/2170010522 >> 172.21.15.37:0/2170010522 conn(0x7ff91c0fc400 msgr2=0x7ff91c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 -- 172.21.15.37:0/2170010522 shutdown_connections 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 -- 172.21.15.37:0/2170010522 wait complete. 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 Processor -- start 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 -- start start 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91c100e90 0x7ff91c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91c101fe0 0x7ff91c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 0x7ff91c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10c9f0 con 0x7ff91c1029a0 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10c870 con 0x7ff91c101fe0 2024-04-23T18:19:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff9236ae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10cb70 con 0x7ff91c100e90 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.579+0000 7ff921c4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 0x7ff91c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 0x7ff91c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58206/0 (socket says 172.21.15.37:58206) 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 -- 172.21.15.37:0/2857793275 learned_addr learned my addr 172.21.15.37:0/2857793275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff92144a640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91c100e90 0x7ff91c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff920c49640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91c101fe0 0x7ff91c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 -- 172.21.15.37:0/2857793275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91c100e90 msgr2=0x7ff91c197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91c100e90 0x7ff91c197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 -- 172.21.15.37:0/2857793275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91c101fe0 msgr2=0x7ff91c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91c101fe0 0x7ff91c197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 -- 172.21.15.37:0/2857793275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c19ed30 con 0x7ff91c1029a0 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff921c4b640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 0x7ff91c19e6c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00cad0 tx=0x7ff90c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00ae20 con 0x7ff91c1029a0 2024-04-23T18:19:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90c005020 con 0x7ff91c1029a0 2024-04-23T18:19:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c19efc0 con 0x7ff91c1029a0 2024-04-23T18:19:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00f6b0 con 0x7ff91c1029a0 2024-04-23T18:19:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.580+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c19f490 con 0x7ff91c1029a0 2024-04-23T18:19:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.581+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91c108270 con 0x7ff91c1029a0 2024-04-23T18:19:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.582+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff90c0051d0 con 0x7ff91c1029a0 2024-04-23T18:19:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.583+0000 7ff91a7fc640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff9040a3c40 0x7ff9040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.583+0000 7ff92144a640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff9040a3c40 0x7ff9040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.583+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff90c0d0a20 con 0x7ff91c1029a0 2024-04-23T18:19:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.583+0000 7ff92144a640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff9040a3c40 0x7ff9040a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff914005fd0 tx=0x7ff91400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.586+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff90c014030 con 0x7ff91c1029a0 2024-04-23T18:19:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.811+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff91c02d5c0 con 0x7ff91c1029a0 2024-04-23T18:19:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.812+0000 7ff91a7fc640 1 -- 172.21.15.37:0/2857793275 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7ff90c09d230 con 0x7ff91c1029a0 2024-04-23T18:19:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:56.805 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:18:40.084107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:19:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.813+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff9040a3c40 msgr2=0x7ff9040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.813+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff9040a3c40 0x7ff9040a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff914005fd0 tx=0x7ff91400b390 comp rx=0 tx=0).stop 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.813+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 msgr2=0x7ff91c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.813+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 0x7ff91c19e6c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00cad0 tx=0x7ff90c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 shutdown_connections 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff9040a3c40 0x7ff9040a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff91c1029a0 0x7ff91c19e6c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91c101fe0 0x7ff91c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 --2- 172.21.15.37:0/2857793275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91c100e90 0x7ff91c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 >> 172.21.15.37:0/2857793275 conn(0x7ff91c0fc400 msgr2=0x7ff91c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 shutdown_connections 2024-04-23T18:19:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.814+0000 7ff9236ae640 1 -- 172.21.15.37:0/2857793275 wait complete. 2024-04-23T18:19:56.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:19:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 -- 172.21.15.37:0/1636210633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24103980 msgr2=0x7f3c2410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1636210633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24103980 0x7f3c2410a210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009ca0 tx=0x7f3c1c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 -- 172.21.15.37:0/1636210633 shutdown_connections 2024-04-23T18:19:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1636210633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24103980 0x7f3c2410a210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1636210633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c24102fc0 0x7f3c24103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1636210633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c24101dd0 0x7f3c241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 -- 172.21.15.37:0/1636210633 >> 172.21.15.37:0/1636210633 conn(0x7f3c240fd360 msgr2=0x7f3c240ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 -- 172.21.15.37:0/1636210633 shutdown_connections 2024-04-23T18:19:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.927+0000 7f3c289ff640 1 -- 172.21.15.37:0/1636210633 wait complete. 2024-04-23T18:19:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 Processor -- start 2024-04-23T18:19:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 -- start start 2024-04-23T18:19:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 0x7f3c24075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c24102fc0 0x7f3c24074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c24103980 0x7f3c24074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2410d950 con 0x7f3c24101dd0 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2410d7d0 con 0x7f3c24103980 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c289ff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2410dad0 con 0x7f3c24102fc0 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c22d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 0x7f3c24075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c22d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 0x7f3c24075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58220/0 (socket says 172.21.15.37:58220) 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c22d9d640 1 -- 172.21.15.37:0/1226692614 learned_addr learned my addr 172.21.15.37:0/1226692614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c2259c640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c24102fc0 0x7f3c24074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.928+0000 7f3c2359e640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c24103980 0x7f3c24074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c22d9d640 1 -- 172.21.15.37:0/1226692614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c24102fc0 msgr2=0x7f3c24074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c22d9d640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c24102fc0 0x7f3c24074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c22d9d640 1 -- 172.21.15.37:0/1226692614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c24103980 msgr2=0x7f3c24074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c22d9d640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c24103980 0x7f3c24074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c22d9d640 1 -- 172.21.15.37:0/1226692614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c24075110 con 0x7f3c24101dd0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c22d9d640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 0x7f3c24075d00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c970 tx=0x7f3c1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14004220 con 0x7f3c24101dd0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c241a20c0 con 0x7f3c24101dd0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c140043c0 con 0x7f3c24101dd0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400f450 con 0x7f3c24101dd0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.929+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c241a2610 con 0x7f3c24101dd0 2024-04-23T18:19:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.930+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c241090f0 con 0x7f3c24101dd0 2024-04-23T18:19:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.931+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3c14003610 con 0x7f3c24101dd0 2024-04-23T18:19:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.932+0000 7f3c0bfff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3c0c0a3c40 0x7f3c0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.932+0000 7f3c2259c640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3c0c0a3c40 0x7f3c0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.932+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3c140d03e0 con 0x7f3c24101dd0 2024-04-23T18:19:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.932+0000 7f3c2259c640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3c0c0a3c40 0x7f3c0c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c24075a70 tx=0x7f3c1800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:56.936+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c14014030 con 0x7f3c24101dd0 2024-04-23T18:19:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.123+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c2402d7f0 con 0x7f3c24101dd0 2024-04-23T18:19:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.124+0000 7f3c0bfff640 1 -- 172.21.15.37:0/1226692614 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22684 (secure 0 0 0) 0x7f3c1409c5a0 con 0x7f3c24101dd0 2024-04-23T18:19:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:57.118 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:19:41.678555+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3c0c0a3c40 msgr2=0x7f3c0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3c0c0a3c40 0x7f3c0c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c24075a70 tx=0x7f3c1800a250 comp rx=0 tx=0).stop 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 msgr2=0x7f3c24075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 0x7f3c24075d00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400c970 tx=0x7f3c1400ce30 comp rx=0 tx=0).stop 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 shutdown_connections 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3c0c0a3c40 0x7f3c0c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c24103980 0x7f3c24074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c24102fc0 0x7f3c24074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 --2- 172.21.15.37:0/1226692614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c24101dd0 0x7f3c24075d00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.126+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 >> 172.21.15.37:0/1226692614 conn(0x7f3c240fd360 msgr2=0x7f3c241062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:57.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.127+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 shutdown_connections 2024-04-23T18:19:57.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.127+0000 7f3c289ff640 1 -- 172.21.15.37:0/1226692614 wait complete. 2024-04-23T18:19:57.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:19:57.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:19:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.243+0000 7f9acde72640 1 -- 172.21.15.37:0/4110286690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 msgr2=0x7f9ac8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.243+0000 7f9acde72640 1 --2- 172.21.15.37:0/4110286690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac8101280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009900 tx=0x7f9ab802a410 comp rx=0 tx=0).stop 2024-04-23T18:19:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- 172.21.15.37:0/4110286690 shutdown_connections 2024-04-23T18:19:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 --2- 172.21.15.37:0/4110286690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac8102a00 0x7f9ac8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 --2- 172.21.15.37:0/4110286690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102040 0x7f9ac81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 --2- 172.21.15.37:0/4110286690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac8101280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- 172.21.15.37:0/4110286690 >> 172.21.15.37:0/4110286690 conn(0x7f9ac80fc420 msgr2=0x7f9ac80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- 172.21.15.37:0/4110286690 shutdown_connections 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- 172.21.15.37:0/4110286690 wait complete. 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 Processor -- start 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- start start 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac810b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac8102040 0x7f9ac8109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102a00 0x7f9ac810a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac810db30 con 0x7f9ac8100e60 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac810d9b0 con 0x7f9ac8102a00 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.244+0000 7f9acde72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac810dcb0 con 0x7f9ac8102040 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac810b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac810b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58240/0 (socket says 172.21.15.37:58240) 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 -- 172.21.15.37:0/2957786840 learned_addr learned my addr 172.21.15.37:0/2957786840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac6ffd640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac8102040 0x7f9ac8109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac7fff640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102a00 0x7f9ac810a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 -- 172.21.15.37:0/2957786840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac8102040 msgr2=0x7f9ac8109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac8102040 0x7f9ac8109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 -- 172.21.15.37:0/2957786840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102a00 msgr2=0x7f9ac810a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102a00 0x7f9ac810a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 -- 172.21.15.37:0/2957786840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac810ac70 con 0x7f9ac8100e60 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac7fff640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102a00 0x7f9ac810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac77fe640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac810b860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8006010 tx=0x7f9ab80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab803f030 con 0x7f9ac8100e60 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab8003cb0 con 0x7f9ac8100e60 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac81a2300 con 0x7f9ac8100e60 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8035070 con 0x7f9ac8100e60 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.245+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac81a2850 con 0x7f9ac8100e60 2024-04-23T18:19:57.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.246+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac81082f0 con 0x7f9ac8100e60 2024-04-23T18:19:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.247+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ab8042030 con 0x7f9ac8100e60 2024-04-23T18:19:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.248+0000 7f9ac4ff9640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3c40 0x7f9ab00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.248+0000 7f9ac6ffd640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3c40 0x7f9ab00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.248+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9ab80f0c40 con 0x7f9ac8100e60 2024-04-23T18:19:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.249+0000 7f9ac6ffd640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3c40 0x7f9ab00a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ac810b5d0 tx=0x7f9abc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.253+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ab80bd450 con 0x7f9ac8100e60 2024-04-23T18:19:57.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:57 smithi172 ceph-mon[32456]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-04-23T18:19:57.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/13385555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:19:57.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/139589428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:19:57.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:57 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2857793275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:57.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:57 smithi173 ceph-mon[32906]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-04-23T18:19:57.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/13385555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:19:57.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/139589428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:19:57.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:57 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2857793275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:57.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:57 smithi037 ceph-mon[29404]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-04-23T18:19:57.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/13385555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:19:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/139589428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:19:57.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:57 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2857793275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:57.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.440+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ac8062b50 con 0x7f9ac8100e60 2024-04-23T18:19:57.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.441+0000 7f9ac4ff9640 1 -- 172.21.15.37:0/2957786840 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f9ab80bcd50 con 0x7f9ac8100e60 2024-04-23T18:19:57.434 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.442+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3c40 msgr2=0x7f9ab00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.442+0000 7f9acde72640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3c40 0x7f9ab00a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ac810b5d0 tx=0x7f9abc00a2d0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.442+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 msgr2=0x7f9ac810b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.442+0000 7f9acde72640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac810b860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8006010 tx=0x7f9ab80042f0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.442+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 shutdown_connections 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9ab00a3c40 0x7f9ab00a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102a00 0x7f9ac810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac8102040 0x7f9ac8109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 --2- 172.21.15.37:0/2957786840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac8100e60 0x7f9ac810b860 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 >> 172.21.15.37:0/2957786840 conn(0x7f9ac80fc420 msgr2=0x7f9ac81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:57.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 shutdown_connections 2024-04-23T18:19:57.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.443+0000 7f9acde72640 1 -- 172.21.15.37:0/2957786840 wait complete. 2024-04-23T18:19:57.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:19:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.558+0000 7fa0cfaae640 1 -- 172.21.15.37:0/4119938923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8103b60 msgr2=0x7fa0c810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.558+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/4119938923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8103b60 0x7fa0c810a3f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009ca0 tx=0x7fa0c002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- 172.21.15.37:0/4119938923 shutdown_connections 2024-04-23T18:19:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/4119938923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8103b60 0x7fa0c810a3f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/4119938923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c81031a0 0x7fa0c8103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/4119938923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- 172.21.15.37:0/4119938923 >> 172.21.15.37:0/4119938923 conn(0x7fa0c80fd520 msgr2=0x7fa0c80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- 172.21.15.37:0/4119938923 shutdown_connections 2024-04-23T18:19:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- 172.21.15.37:0/4119938923 wait complete. 2024-04-23T18:19:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 Processor -- start 2024-04-23T18:19:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- start start 2024-04-23T18:19:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c81031a0 0x7fa0c81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c8103b60 0x7fa0c819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c810db30 con 0x7fa0c8101fb0 2024-04-23T18:19:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c810d9b0 con 0x7fa0c81031a0 2024-04-23T18:19:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cfaae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c810dcb0 con 0x7fa0c8103b60 2024-04-23T18:19:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.559+0000 7fa0cd84a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58252/0 (socket says 172.21.15.37:58252) 2024-04-23T18:19:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 -- 172.21.15.37:0/2250640046 learned_addr learned my addr 172.21.15.37:0/2250640046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 -- 172.21.15.37:0/2250640046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c8103b60 msgr2=0x7fa0c819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c8103b60 0x7fa0c819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd049640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c81031a0 0x7fa0c81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 -- 172.21.15.37:0/2250640046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c81031a0 msgr2=0x7fa0c81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c81031a0 0x7fa0c81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 -- 172.21.15.37:0/2250640046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c819fdc0 con 0x7fa0c8101fb0 2024-04-23T18:19:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd049640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c81031a0 0x7fa0c81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cd84a640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81981a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d770 tx=0x7fa0c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4017070 con 0x7fa0c8101fb0 2024-04-23T18:19:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c81a00b0 con 0x7fa0c8101fb0 2024-04-23T18:19:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4009c30 con 0x7fa0c8101fb0 2024-04-23T18:19:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4003cc0 con 0x7fa0c8101fb0 2024-04-23T18:19:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.560+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81a0680 con 0x7fa0c8101fb0 2024-04-23T18:19:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.561+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c81092d0 con 0x7fa0c8101fb0 2024-04-23T18:19:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.562+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0c4003e60 con 0x7fa0c8101fb0 2024-04-23T18:19:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.563+0000 7fa0beffd640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0b40a3c40 0x7fa0b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.563+0000 7fa0cd049640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0b40a3c40 0x7fa0b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.563+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa0c40d0480 con 0x7fa0c8101fb0 2024-04-23T18:19:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.564+0000 7fa0cd049640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0b40a3c40 0x7fa0b40a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8006fd0 tx=0x7fa0b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.567+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0c4009de0 con 0x7fa0c8101fb0 2024-04-23T18:19:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.752+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0c8062bf0 con 0x7fa0c8101fb0 2024-04-23T18:19:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.753+0000 7fa0beffd640 1 -- 172.21.15.37:0/2250640046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fa0c409cc90 con 0x7fa0c8101fb0 2024-04-23T18:19:57.746 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:19:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0b40a3c40 msgr2=0x7fa0b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0b40a3c40 0x7fa0b40a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8006fd0 tx=0x7fa0b8008040 comp rx=0 tx=0).stop 2024-04-23T18:19:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 msgr2=0x7fa0c81981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81981a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400d770 tx=0x7fa0c400dc30 comp rx=0 tx=0).stop 2024-04-23T18:19:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 shutdown_connections 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa0b40a3c40 0x7fa0b40a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c8103b60 0x7fa0c819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0c81031a0 0x7fa0c81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 --2- 172.21.15.37:0/2250640046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c8101fb0 0x7fa0c81981a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 >> 172.21.15.37:0/2250640046 conn(0x7fa0c80fd520 msgr2=0x7fa0c81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 shutdown_connections 2024-04-23T18:19:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.755+0000 7fa0cfaae640 1 -- 172.21.15.37:0/2250640046 wait complete. 2024-04-23T18:19:57.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:19:57.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 -- 172.21.15.37:0/1568043932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 msgr2=0x7f801010c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 --2- 172.21.15.37:0/1568043932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f801010c2e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8000009ca0 tx=0x7f800002a950 comp rx=0 tx=0).stop 2024-04-23T18:19:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 -- 172.21.15.37:0/1568043932 shutdown_connections 2024-04-23T18:19:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 --2- 172.21.15.37:0/1568043932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f801010c2e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 --2- 172.21.15.37:0/1568043932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 0x7f8010074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 --2- 172.21.15.37:0/1568043932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8010075f30 0x7f8010074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 -- 172.21.15.37:0/1568043932 >> 172.21.15.37:0/1568043932 conn(0x7f80100fd520 msgr2=0x7f80100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 -- 172.21.15.37:0/1568043932 shutdown_connections 2024-04-23T18:19:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.871+0000 7f8015568640 1 -- 172.21.15.37:0/1568043932 wait complete. 2024-04-23T18:19:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 Processor -- start 2024-04-23T18:19:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 -- start start 2024-04-23T18:19:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 0x7f801019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8010075f30 0x7f801019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f80101a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f801010fcf0 con 0x7f8010109e40 2024-04-23T18:19:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f801010fb70 con 0x7f8010075f30 2024-04-23T18:19:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f8015568640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f801010fe70 con 0x7f8010074950 2024-04-23T18:19:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.872+0000 7f800f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f80101a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f80101a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58266/0 (socket says 172.21.15.37:58266) 2024-04-23T18:19:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 -- 172.21.15.37:0/259302088 learned_addr learned my addr 172.21.15.37:0/259302088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800e7fc640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8010075f30 0x7f801019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800effd640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 0x7f801019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 -- 172.21.15.37:0/259302088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 msgr2=0x7f801019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 0x7f801019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 -- 172.21.15.37:0/259302088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8010075f30 msgr2=0x7f801019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8010075f30 0x7f801019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 -- 172.21.15.37:0/259302088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101a1ee0 con 0x7f8010109e40 2024-04-23T18:19:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800effd640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 0x7f801019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:57.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f800f7fe640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f80101a1870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8000006010 tx=0x7f800002fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:57.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000034040 con 0x7f8010109e40 2024-04-23T18:19:57.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80000308f0 con 0x7f8010109e40 2024-04-23T18:19:57.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101a2170 con 0x7f8010109e40 2024-04-23T18:19:57.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000039450 con 0x7f8010109e40 2024-04-23T18:19:57.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.873+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101a2660 con 0x7f8010109e40 2024-04-23T18:19:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.874+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010108c00 con 0x7f8010109e40 2024-04-23T18:19:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.875+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8000037020 con 0x7f8010109e40 2024-04-23T18:19:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.876+0000 7f8007fff640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7ffc0a3c40 0x7f7ffc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.876+0000 7f800effd640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7ffc0a3c40 0x7f7ffc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.876+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f80000f1750 con 0x7f8010109e40 2024-04-23T18:19:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.876+0000 7f800effd640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7ffc0a3c40 0x7f7ffc0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8008009b40 tx=0x7f8008005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:57.880+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80000be0d0 con 0x7f8010109e40 2024-04-23T18:19:58.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.073+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8010062b50 con 0x7f8010109e40 2024-04-23T18:19:58.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.073+0000 7f8007fff640 1 -- 172.21.15.37:0/259302088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f8000041e30 con 0x7f8010109e40 2024-04-23T18:19:58.067 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:19:58.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7ffc0a3c40 msgr2=0x7f7ffc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7ffc0a3c40 0x7f7ffc0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8008009b40 tx=0x7f8008005e50 comp rx=0 tx=0).stop 2024-04-23T18:19:58.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 msgr2=0x7f80101a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f80101a1870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8000006010 tx=0x7f800002fc00 comp rx=0 tx=0).stop 2024-04-23T18:19:58.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 shutdown_connections 2024-04-23T18:19:58.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7ffc0a3c40 0x7f7ffc0a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8010109e40 0x7f80101a1870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8010075f30 0x7f801019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 --2- 172.21.15.37:0/259302088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8010074950 0x7f801019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 >> 172.21.15.37:0/259302088 conn(0x7f80100fd520 msgr2=0x7f8010105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:58.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 shutdown_connections 2024-04-23T18:19:58.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.076+0000 7f8015568640 1 -- 172.21.15.37:0/259302088 wait complete. 2024-04-23T18:19:58.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:19:58.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 -- 172.21.15.37:0/3449084583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 msgr2=0x7f65941036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3449084583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f65941036c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6588009930 tx=0x7f658802a420 comp rx=0 tx=0).stop 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 -- 172.21.15.37:0/3449084583 shutdown_connections 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3449084583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6594103c00 0x7f659410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3449084583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f65941036c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3449084583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594100270 0x7f6594100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 -- 172.21.15.37:0/3449084583 >> 172.21.15.37:0/3449084583 conn(0x7f65940fbbc0 msgr2=0x7f65940fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 -- 172.21.15.37:0/3449084583 shutdown_connections 2024-04-23T18:19:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.189+0000 7f6598ae8640 1 -- 172.21.15.37:0/3449084583 wait complete. 2024-04-23T18:19:58.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 Processor -- start 2024-04-23T18:19:58.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 -- start start 2024-04-23T18:19:58.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594100270 0x7f659419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f659419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6594103c00 0x7f65941a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f659410db10 con 0x7f6594103240 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f659410d990 con 0x7f6594100270 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6598ae8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f659410dc90 con 0x7f6594103c00 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f659419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f659419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58288/0 (socket says 172.21.15.37:58288) 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 -- 172.21.15.37:0/3276956268 learned_addr learned my addr 172.21.15.37:0/3276956268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6592d9d640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6594103c00 0x7f65941a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f659259c640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594100270 0x7f659419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 -- 172.21.15.37:0/3276956268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6594103c00 msgr2=0x7f65941a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6594103c00 0x7f65941a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 -- 172.21.15.37:0/3276956268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594100270 msgr2=0x7f659419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594100270 0x7f659419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.190+0000 7f6591d9b640 1 -- 172.21.15.37:0/3276956268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65941a2040 con 0x7f6594103240 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.191+0000 7f6591d9b640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f659419a960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6588009a50 tx=0x7f6588032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.191+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658803f030 con 0x7f6594103240 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.191+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65880044b0 con 0x7f6594103240 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.191+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65941a22d0 con 0x7f6594103240 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.191+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588035070 con 0x7f6594103240 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.191+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a2810 con 0x7f6594103240 2024-04-23T18:19:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.192+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65941948e0 con 0x7f6594103240 2024-04-23T18:19:58.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.193+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6588042030 con 0x7f6594103240 2024-04-23T18:19:58.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.194+0000 7f65837fe640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f657c0a3cc0 0x7f657c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.194+0000 7f659259c640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f657c0a3cc0 0x7f657c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.194+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f65880f1150 con 0x7f6594103240 2024-04-23T18:19:58.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.194+0000 7f659259c640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f657c0a3cc0 0x7f657c0a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6584005fd0 tx=0x7f6584005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:58.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.197+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65880bd960 con 0x7f6594103240 2024-04-23T18:19:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:58 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1226692614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:58 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2957786840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:58 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2250640046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:58 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/259302088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:58 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1226692614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:58.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:58 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2957786840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:58 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2250640046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:58 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/259302088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1226692614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2957786840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2250640046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:58 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/259302088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:19:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.423+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f659402d5c0 con 0x7f6594103240 2024-04-23T18:19:58.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.424+0000 7f65837fe640 1 -- 172.21.15.37:0/3276956268 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f65880bd260 con 0x7f6594103240 2024-04-23T18:19:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:58.417 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:18:40.084107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:19:58.417 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:19:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f657c0a3cc0 msgr2=0x7f657c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f657c0a3cc0 0x7f657c0a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6584005fd0 tx=0x7f6584005850 comp rx=0 tx=0).stop 2024-04-23T18:19:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 msgr2=0x7f659419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f659419a960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6588009a50 tx=0x7f6588032f90 comp rx=0 tx=0).stop 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 shutdown_connections 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f657c0a3cc0 0x7f657c0a6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6594103c00 0x7f65941a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6594103240 0x7f659419a960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 --2- 172.21.15.37:0/3276956268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594100270 0x7f659419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 >> 172.21.15.37:0/3276956268 conn(0x7f65940fbbc0 msgr2=0x7f6594108b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 shutdown_connections 2024-04-23T18:19:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.426+0000 7f6598ae8640 1 -- 172.21.15.37:0/3276956268 wait complete. 2024-04-23T18:19:58.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 -- 172.21.15.37:0/4145383058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888103b50 msgr2=0x7fc88810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 --2- 172.21.15.37:0/4145383058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888103b50 0x7fc88810a3e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ca0 tx=0x7fc88002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 -- 172.21.15.37:0/4145383058 shutdown_connections 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 --2- 172.21.15.37:0/4145383058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888103b50 0x7fc88810a3e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 --2- 172.21.15.37:0/4145383058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc888103190 0x7fc888103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 --2- 172.21.15.37:0/4145383058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888101fa0 0x7fc8881023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 -- 172.21.15.37:0/4145383058 >> 172.21.15.37:0/4145383058 conn(0x7fc8880fd530 msgr2=0x7fc8880ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 -- 172.21.15.37:0/4145383058 shutdown_connections 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 -- 172.21.15.37:0/4145383058 wait complete. 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 Processor -- start 2024-04-23T18:19:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.540+0000 7fc88f35c640 1 -- start start 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 0x7fc888075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888103190 0x7fc888074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc888103b50 0x7fc888074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810dae0 con 0x7fc888101fa0 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810d960 con 0x7fc888103190 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810dc60 con 0x7fc888103b50 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 0x7fc888075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 0x7fc888075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58304/0 (socket says 172.21.15.37:58304) 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 -- 172.21.15.37:0/1922225823 learned_addr learned my addr 172.21.15.37:0/1922225823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88c8f7640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888103190 0x7fc888074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d8f9640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc888103b50 0x7fc888074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 -- 172.21.15.37:0/1922225823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc888103b50 msgr2=0x7fc888074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc888103b50 0x7fc888074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 -- 172.21.15.37:0/1922225823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888103190 msgr2=0x7fc888074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888103190 0x7fc888074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 -- 172.21.15.37:0/1922225823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc888075110 con 0x7fc888101fa0 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88c8f7640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888103190 0x7fc888074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88d0f8640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 0x7fc888075d00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc88400d770 tx=0x7fc88400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884017070 con 0x7fc888101fa0 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc884009c30 con 0x7fc888101fa0 2024-04-23T18:19:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884003cc0 con 0x7fc888101fa0 2024-04-23T18:19:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8881a22a0 con 0x7fc888101fa0 2024-04-23T18:19:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.541+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881a2840 con 0x7fc888101fa0 2024-04-23T18:19:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.542+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8881092c0 con 0x7fc888101fa0 2024-04-23T18:19:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.543+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc884003e60 con 0x7fc888101fa0 2024-04-23T18:19:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.544+0000 7fc87e7fc640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc8740a3cc0 0x7fc8740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.544+0000 7fc88c8f7640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc8740a3cc0 0x7fc8740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.544+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc8840cff60 con 0x7fc888101fa0 2024-04-23T18:19:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.544+0000 7fc88c8f7640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc8740a3cc0 0x7fc8740a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc888075a70 tx=0x7fc878005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:58.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.549+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc884009de0 con 0x7fc888101fa0 2024-04-23T18:19:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.735+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc888062b50 con 0x7fc888101fa0 2024-04-23T18:19:58.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.736+0000 7fc87e7fc640 1 -- 172.21.15.37:0/1922225823 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22684 (secure 0 0 0) 0x7fc88409c7d0 con 0x7fc888101fa0 2024-04-23T18:19:58.729 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:58.730 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:19:41.678555+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.738+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc8740a3cc0 msgr2=0x7fc8740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.738+0000 7fc88f35c640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc8740a3cc0 0x7fc8740a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc888075a70 tx=0x7fc878005f90 comp rx=0 tx=0).stop 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.738+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 msgr2=0x7fc888075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.738+0000 7fc88f35c640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 0x7fc888075d00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc88400d770 tx=0x7fc88400dc30 comp rx=0 tx=0).stop 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 shutdown_connections 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc8740a3cc0 0x7fc8740a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc888103b50 0x7fc888074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc888103190 0x7fc888074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 --2- 172.21.15.37:0/1922225823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc888101fa0 0x7fc888075d00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 >> 172.21.15.37:0/1922225823 conn(0x7fc8880fd530 msgr2=0x7fc888106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 shutdown_connections 2024-04-23T18:19:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.739+0000 7fc88f35c640 1 -- 172.21.15.37:0/1922225823 wait complete. 2024-04-23T18:19:58.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 -- 172.21.15.37:0/479536275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac105360 msgr2=0x7f71ac107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 --2- 172.21.15.37:0/479536275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac105360 0x7f71ac107750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009900 tx=0x7f71a002a410 comp rx=0 tx=0).stop 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 -- 172.21.15.37:0/479536275 shutdown_connections 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 --2- 172.21.15.37:0/479536275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71ac107c90 0x7f71ac10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 --2- 172.21.15.37:0/479536275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac105360 0x7f71ac107750 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 --2- 172.21.15.37:0/479536275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 0x7f71ac104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 -- 172.21.15.37:0/479536275 >> 172.21.15.37:0/479536275 conn(0x7f71ac0fc440 msgr2=0x7f71ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:58.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 -- 172.21.15.37:0/479536275 shutdown_connections 2024-04-23T18:19:58.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 -- 172.21.15.37:0/479536275 wait complete. 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 Processor -- start 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.852+0000 7f71b0f87640 1 -- start start 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71b0f87640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 0x7f71ac198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71b0f87640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71ac105360 0x7f71ac1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71b0f87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 0x7f71ac19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71b0f87640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10daf0 con 0x7f71ac107c90 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71b0f87640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10d970 con 0x7f71ac105360 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71b0f87640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10dc70 con 0x7f71ac102a30 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 0x7f71ac19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aa59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 0x7f71ac198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 0x7f71ac19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58310/0 (socket says 172.21.15.37:58310) 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 -- 172.21.15.37:0/4288753665 learned_addr learned my addr 172.21.15.37:0/4288753665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:58.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71a9d9b640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71ac105360 0x7f71ac1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 -- 172.21.15.37:0/4288753665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 msgr2=0x7f71ac198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 0x7f71ac198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 -- 172.21.15.37:0/4288753665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71ac105360 msgr2=0x7f71ac1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71ac105360 0x7f71ac1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 -- 172.21.15.37:0/4288753665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac19feb0 con 0x7f71ac107c90 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aa59c640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 0x7f71ac198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f71aad9d640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 0x7f71ac19f840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009fd0 tx=0x7f71a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.853+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4017070 con 0x7f71ac107c90 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.854+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac1a01a0 con 0x7f71ac107c90 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.854+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71a4003e20 con 0x7f71ac107c90 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.854+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4010460 con 0x7f71ac107c90 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.854+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1a0670 con 0x7f71ac107c90 2024-04-23T18:19:58.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.855+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac0fdf30 con 0x7f71ac107c90 2024-04-23T18:19:58.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.856+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71a4003970 con 0x7f71ac107c90 2024-04-23T18:19:58.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.857+0000 7f719b7fe640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f71940a3c40 0x7f71940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:58.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.857+0000 7f71aa59c640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f71940a3c40 0x7f71940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:58.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.857+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f71a40d02a0 con 0x7f71ac107c90 2024-04-23T18:19:58.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.857+0000 7f71aa59c640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f71940a3c40 0x7f71940a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f719c009fc0 tx=0x7f719c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:58.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:58.861+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71a409c4c0 con 0x7f71ac107c90 2024-04-23T18:19:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.086+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71ac062b50 con 0x7f71ac107c90 2024-04-23T18:19:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.087+0000 7f719b7fe640 1 -- 172.21.15.37:0/4288753665 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f71a40a07c0 con 0x7f71ac107c90 2024-04-23T18:19:59.080 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:19:59.080 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:18:40.084107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:19:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f71940a3c40 msgr2=0x7f71940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f71940a3c40 0x7f71940a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f719c009fc0 tx=0x7f719c005f90 comp rx=0 tx=0).stop 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 msgr2=0x7f71ac19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 0x7f71ac19f840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009fd0 tx=0x7f71a400b260 comp rx=0 tx=0).stop 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 shutdown_connections 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f71940a3c40 0x7f71940a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107c90 0x7f71ac19f840 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71ac105360 0x7f71ac1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 --2- 172.21.15.37:0/4288753665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ac102a30 0x7f71ac198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 >> 172.21.15.37:0/4288753665 conn(0x7f71ac0fc440 msgr2=0x7f71ac1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 shutdown_connections 2024-04-23T18:19:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.089+0000 7f71b0f87640 1 -- 172.21.15.37:0/4288753665 wait complete. 2024-04-23T18:19:59.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:18:34.408115+0000', 'modified': '2024-04-23T18:18:40.084107+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24484, 'mds_2': 24479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2640036963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2640036963}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2640036963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2419231317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2419231317}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2419231317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6837/3035435067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3035435067}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3035435067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.172:6837/364708999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 364708999}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 364708999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.173:6839/496716250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 496716250}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 496716250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6839/2128854391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 2128854391}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 2128854391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24479, 24484, 24509]} 2024-04-23T18:19:59.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-23T18:19:59.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:19:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 -- 172.21.15.37:0/2397065995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b60101fc0 msgr2=0x7f2b601023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/2397065995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b60101fc0 0x7f2b601023e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009900 tx=0x7f2b5002a410 comp rx=0 tx=0).stop 2024-04-23T18:19:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 -- 172.21.15.37:0/2397065995 shutdown_connections 2024-04-23T18:19:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/2397065995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b60103b70 0x7f2b6010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/2397065995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b601031b0 0x7f2b60103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/2397065995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b60101fc0 0x7f2b601023e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 -- 172.21.15.37:0/2397065995 >> 172.21.15.37:0/2397065995 conn(0x7f2b600fd530 msgr2=0x7f2b600ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 -- 172.21.15.37:0/2397065995 shutdown_connections 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.204+0000 7f2b64ca0640 1 -- 172.21.15.37:0/2397065995 wait complete. 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 Processor -- start 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 -- start start 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b60101fc0 0x7f2b60198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 0x7f2b601987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b60103b70 0x7f2b6019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010db60 con 0x7f2b601031b0 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010d9e0 con 0x7f2b60101fc0 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b64ca0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010dce0 con 0x7f2b60103b70 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 0x7f2b601987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 0x7f2b601987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58322/0 (socket says 172.21.15.37:58322) 2024-04-23T18:19:59.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b57fff640 1 -- 172.21.15.37:0/1924700251 learned_addr learned my addr 172.21.15.37:0/1924700251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b60101fc0 0x7f2b60198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.205+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b60103b70 0x7f2b6019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b57fff640 1 -- 172.21.15.37:0/1924700251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b60103b70 msgr2=0x7f2b6019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b57fff640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b60103b70 0x7f2b6019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b57fff640 1 -- 172.21.15.37:0/1924700251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b60101fc0 msgr2=0x7f2b60198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b57fff640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b60101fc0 0x7f2b60198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b57fff640 1 -- 172.21.15.37:0/1924700251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6019fe80 con 0x7f2b601031b0 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b60101fc0 0x7f2b60198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b57fff640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 0x7f2b601987a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800b4a0 tx=0x7f2b4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b577fe640 1 -- 172.21.15.37:0/1924700251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48014070 con 0x7f2b601031b0 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b601a0170 con 0x7f2b601031b0 2024-04-23T18:19:59.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b577fe640 1 -- 172.21.15.37:0/1924700251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b48010ac0 con 0x7f2b601031b0 2024-04-23T18:19:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b577fe640 1 -- 172.21.15.37:0/1924700251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48017c30 con 0x7f2b601031b0 2024-04-23T18:19:59.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.206+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b601a0790 con 0x7f2b601031b0 2024-04-23T18:19:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b577fe640 1 -- 172.21.15.37:0/1924700251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2b48017070 con 0x7f2b601031b0 2024-04-23T18:19:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 Processor -- start 2024-04-23T18:19:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 -- start start 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd649c50 con 0x55bdcd6e8f90 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd5e6830 con 0x55bdcd6ea160 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b679bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd645c10 con 0x55bdcd6eab20 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.209+0000 7f2b5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6e93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58334/0 (socket says 172.21.15.37:58334) 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 -- 172.21.15.37:0/2694524152 learned_addr learned my addr 172.21.15.37:0/2694524152 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b577fe640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2b4c0a3c40 0x7f2b4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b57fff640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 -- 172.21.15.37:0/2694524152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 msgr2=0x55bdcd6f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 -- 172.21.15.37:0/2694524152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 msgr2=0x55bdcd6ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6ea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 -- 172.21.15.37:0/2694524152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdcd35c2d0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b577fe640 1 -- 172.21.15.37:0/1924700251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f2b480cfc90 con 0x7f2b601031b0 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b57fff640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2b4c0a3c40 0x7f2b4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6e93b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b580103f0 tx=0x7f2b580108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723727093cece2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b54ff9640 1 -- 172.21.15.37:0/2694524152 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5801c9c0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2b4c0a3c40 0x7f2b4c0a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2b58020000 tx=0x7f2b5801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b54ff9640 1 -- 172.21.15.37:0/2694524152 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5801cb60 con 0x55bdcd6e8f90 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.210+0000 7f2b54ff9640 1 -- 172.21.15.37:0/2694524152 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5801b3d0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 -- 172.21.15.37:0/2694524152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 msgr2=0x55bdcd6e93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6e93b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b580103f0 tx=0x7f2b580108b0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 -- 172.21.15.37:0/2694524152 shutdown_connections 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2694524152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6e93b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 -- 172.21.15.37:0/2694524152 >> 172.21.15.37:0/2694524152 conn(0x55bdcd6e0070 msgr2=0x55bdcd6e0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 -- 172.21.15.37:0/2694524152 shutdown_connections 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 -- 172.21.15.37:0/2694524152 wait complete. 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.211+0000 7f2b679bb740 1 -- start start 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 Processor -- start 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6fa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd6f1b30 con 0x55bdcd6ea160 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd5e4470 con 0x55bdcd6eab20 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd6f1cb0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b679bb740 10 client.0 _ll_register_callbacks cb 0x55bdcd609f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58342/0 (socket says 172.21.15.37:58342) 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 -- 172.21.15.37:0/2172904277 learned_addr learned my addr 172.21.15.37:0/2172904277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 -- 172.21.15.37:0/2172904277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6e8f90 msgr2=0x55bdcd6f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 -- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 msgr2=0x55bdcd6fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6fa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 -- 172.21.15.37:0/2172904277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdcd4266b0 con 0x55bdcd6ea160 2024-04-23T18:19:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b57fff640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f38f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2b480cd710 tx=0x7f2b4806b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2724e9c6344c90af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.212+0000 7f2b45ffb640 1 -- 172.21.15.37:0/2172904277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801e040 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5801cb60 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b45ffb640 1 -- 172.21.15.37:0/2172904277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f2b48098520 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b45ffb640 1 -- 172.21.15.37:0/2172904277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2b4809c5b0 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 10 client.15042 fetch_fsmap learned FSMap version 22 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bdcd6fdcd0 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b45ffb640 1 -- 172.21.15.37:0/2172904277 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5801cb60 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b45ffb640 1 -- 172.21.15.37:0/2172904277 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2b480cd820 con 0x55bdcd6ea160 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 10 client.15042 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 10 client.15042 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 20 client.15042 populate_metadata read hostname 'smithi037' 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd7002a0 0x55bdcd701710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 4 client.15042 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:19:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdcd657670 con 0x55bdcd7002a0 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.213+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd7002a0 0x55bdcd701710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.214+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd7002a0 0x55bdcd701710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e0e2e9051efc487 server_cookie=81fa28909513fc23 in_seq=0 out_seq=0 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.214+0000 7f2b45ffb640 10 client.15042 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b45ffb640 1 -- 172.21.15.37:0/2172904277 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdcd657670 con 0x55bdcd7002a0 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b45ffb640 10 client.15042 handle_command_reply: tid=0 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 client.15042 shutdown 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 msgr2=0x55bdcd6f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f38f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2b480cd710 tx=0x7f2b4806b7e0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 shutdown_connections 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd7002a0 0x55bdcd701710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b45ffb640 0 client.15042 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:19:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f38f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 --2- 172.21.15.37:0/2172904277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 >> 172.21.15.37:0/2172904277 conn(0x55bdcd6e0070 msgr2=0x55bdcd6df1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 shutdown_connections 2024-04-23T18:19:59.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 1 -- 172.21.15.37:0/2172904277 wait complete. 2024-04-23T18:19:59.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.215+0000 7f2b679bb740 20 client.15042 trim_cache size 0 max 16384 2024-04-23T18:19:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.233+0000 7f2b679bb740 1 Processor -- start 2024-04-23T18:19:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.233+0000 7f2b679bb740 1 -- start start 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6ea160 0x55bdcd6fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b679bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd6ffa50 con 0x55bdcd6e8f90 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b679bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd700120 con 0x55bdcd6eab20 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b679bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd7007f0 con 0x55bdcd6ea160 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6de940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58354/0 (socket says 172.21.15.37:58354) 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 -- 172.21.15.37:0/4268061252 learned_addr learned my addr 172.21.15.37:0/4268061252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 -- 172.21.15.37:0/4268061252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6ea160 msgr2=0x55bdcd6fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6ea160 0x55bdcd6fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 -- 172.21.15.37:0/4268061252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 msgr2=0x55bdcd6dee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 -- 172.21.15.37:0/4268061252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdcd6f0ea0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6de940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005ab0 tx=0x7f2b50005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4892294dc590472 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b47fff640 1 -- 172.21.15.37:0/4268061252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50035040 con 0x55bdcd6e8f90 2024-04-23T18:19:59.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b47fff640 1 -- 172.21.15.37:0/4268061252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b500053f0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.234+0000 7f2b47fff640 1 -- 172.21.15.37:0/4268061252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50031700 con 0x55bdcd6e8f90 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- 172.21.15.37:0/4268061252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 msgr2=0x55bdcd6de940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6de940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005ab0 tx=0x7f2b50005a40 comp rx=0 tx=0).stop 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- 172.21.15.37:0/4268061252 shutdown_connections 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6eab20 0x55bdcd6dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6ea160 0x55bdcd6fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- 172.21.15.37:0/4268061252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6de940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- 172.21.15.37:0/4268061252 >> 172.21.15.37:0/4268061252 conn(0x55bdcd6e0070 msgr2=0x55bdcd6db0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- 172.21.15.37:0/4268061252 shutdown_connections 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- 172.21.15.37:0/4268061252 wait complete. 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- start start 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 Processor -- start 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd701590 con 0x55bdcd6e8f90 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd700ec0 con 0x55bdcd6ea160 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bdcd701c60 con 0x55bdcd6eab20 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b679bb740 10 client.0 _ll_register_callbacks cb 0x7f2b48017550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58360/0 (socket says 172.21.15.37:58360) 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.235+0000 7f2b5e59c640 1 -- 172.21.15.37:0/378729463 learned_addr learned my addr 172.21.15.37:0/378729463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5e59c640 1 -- 172.21.15.37:0/378729463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 msgr2=0x55bdcd6f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5e59c640 1 -- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 msgr2=0x55bdcd6f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5e59c640 1 -- 172.21.15.37:0/378729463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdcd6f99b0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5ed9d640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b5e59c640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f5e60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009230 tx=0x7f2b500042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6fc18c9e6b6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b45ffb640 1 -- 172.21.15.37:0/378729463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50050020 con 0x55bdcd6e8f90 2024-04-23T18:19:59.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b45ffb640 1 -- 172.21.15.37:0/378729463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f2b50048040 con 0x55bdcd6e8f90 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b500053f0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b45ffb640 1 -- 172.21.15.37:0/378729463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2b50035030 con 0x55bdcd6e8f90 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b679bb740 10 client.15048 fetch_fsmap learned FSMap version 22 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bdcd6facb0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.236+0000 7f2b45ffb640 1 -- 172.21.15.37:0/378729463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b500053f0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b45ffb640 1 -- 172.21.15.37:0/378729463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2b500046c0 con 0x55bdcd6e8f90 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 10 client.15048 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:19:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 20 client.15048 populate_metadata read hostname 'smithi037' 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd76a8d0 0x55bdcd766d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 4 client.15048 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "status"}) -- 0x55bdcd6514e0 con 0x55bdcd76a8d0 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b57fff640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd76a8d0 0x55bdcd766d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b45ffb640 10 client.15048 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b57fff640 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd76a8d0 0x55bdcd766d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5630b8a3d892aa6a server_cookie=2451f70cf11db950 in_seq=0 out_seq=0 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b45ffb640 1 -- 172.21.15.37:0/378729463 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bdcd6514e0 con 0x55bdcd76a8d0 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b45ffb640 10 client.15048 handle_command_reply: tid=0 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.237+0000 7f2b679bb740 1 client.15048 shutdown 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 msgr2=0x55bdcd6f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f5e60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009230 tx=0x7f2b500042a0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 shutdown_connections 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x55bdcd76a8d0 0x55bdcd766d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bdcd6eab20 0x55bdcd6f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdcd6ea160 0x55bdcd6f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b45ffb640 0 client.15048 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 --2- 172.21.15.37:0/378729463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdcd6e8f90 0x55bdcd6f5e60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 >> 172.21.15.37:0/378729463 conn(0x55bdcd6e0070 msgr2=0x55bdcd6e3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 shutdown_connections 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 1 -- 172.21.15.37:0/378729463 wait complete. 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b679bb740 20 client.15048 trim_cache size 0 max 16384 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:19:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24479, 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 80.163980086999999, 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 101.691842886 2024-04-23T18:19:59.244 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2b4c0a3c40 msgr2=0x7f2b4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2b4c0a3c40 0x7f2b4c0a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2b58020000 tx=0x7f2b5801d000 comp rx=0 tx=0).stop 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 msgr2=0x7f2b601987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 0x7f2b601987a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800b4a0 tx=0x7f2b4800b960 comp rx=0 tx=0).stop 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 shutdown_connections 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f2b4c0a3c40 0x7f2b4c0a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b60103b70 0x7f2b6019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601031b0 0x7f2b601987a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 --2- 172.21.15.37:0/1924700251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b60101fc0 0x7f2b60198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 >> 172.21.15.37:0/1924700251 conn(0x7f2b600fd530 msgr2=0x7f2b60106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 shutdown_connections 2024-04-23T18:19:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.238+0000 7f2b64ca0640 1 -- 172.21.15.37:0/1924700251 wait complete. 2024-04-23T18:19:59.247 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:19:59.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:59 smithi172 ceph-mon[32456]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:59.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:59 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3276956268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:59 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1922225823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:19:59 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4288753665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.356+0000 7f3aae0cb640 1 -- 172.21.15.37:0/1724004721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8102a10 msgr2=0x7f3aa8104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.356+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/1724004721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8102a10 0x7f3aa8104e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009900 tx=0x7f3a9802a3f0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 -- 172.21.15.37:0/1724004721 shutdown_connections 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/1724004721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8107c70 0x7f3aa810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/1724004721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3aa8105340 0x7f3aa8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/1724004721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8102a10 0x7f3aa8104e00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 -- 172.21.15.37:0/1724004721 >> 172.21.15.37:0/1724004721 conn(0x7f3aa80fc400 msgr2=0x7f3aa80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 -- 172.21.15.37:0/1724004721 shutdown_connections 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 -- 172.21.15.37:0/1724004721 wait complete. 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 Processor -- start 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.357+0000 7f3aae0cb640 1 -- start start 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3aa8105340 0x7f3aa81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810dac0 con 0x7f3aa8107c70 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810d940 con 0x7f3aa8102a10 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810dc40 con 0x7f3aa8105340 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58380/0 (socket says 172.21.15.37:58380) 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 -- 172.21.15.37:0/736336794 learned_addr learned my addr 172.21.15.37:0/736336794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3aa8105340 0x7f3aa81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 -- 172.21.15.37:0/736336794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3aa8105340 msgr2=0x7f3aa81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3aa8105340 0x7f3aa81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 -- 172.21.15.37:0/736336794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8102a10 msgr2=0x7f3aa8198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 -- 172.21.15.37:0/736336794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa819fd90 con 0x7f3aa8107c70 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000eaa0 tx=0x7f3aa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/736336794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cdb0 con 0x7f3aa8107c70 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81a0080 con 0x7f3aa8107c70 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/736336794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa0005020 con 0x7f3aa8107c70 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/736336794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00116d0 con 0x7f3aa8107c70 2024-04-23T18:19:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.358+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa81a05d0 con 0x7f3aa8107c70 2024-04-23T18:19:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:59 smithi173 ceph-mon[32906]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:59 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3276956268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:59 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1922225823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:19:59 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4288753665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/736336794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3aa00051d0 con 0x7f3aa8107c70 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 Processor -- start 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 -- start start 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df0835fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df08371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df083dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555df079ac50 con 0x555df0837710 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555df0737830 con 0x555df0835b80 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555df0796c10 con 0x555df0836d50 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df083dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.361+0000 7f3aa4ff9640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a900a3c40 0x7f3a900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df083dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58382/0 (socket says 172.21.15.37:58382) 2024-04-23T18:19:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/3308796100 learned_addr learned my addr 172.21.15.37:0/3308796100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df08371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df0835fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a900a3c40 0x7f3a900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/736336794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3aa00d08c0 con 0x7f3aa8107c70 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3308796100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 msgr2=0x555df0835fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df0835fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3308796100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 msgr2=0x555df083dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df083dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3308796100 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df04ad2d0 con 0x555df0836d50 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df083dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a900a3c40 0x7f3a900a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000efd0 tx=0x7f3aa0087040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df08371d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009230 tx=0x7f3a98032e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e23fb40dff9a6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3a8dffb640 1 -- 172.21.15.37:0/3308796100 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9803f070 con 0x555df0836d50 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3a8dffb640 1 -- 172.21.15.37:0/3308796100 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a980317a0 con 0x555df0836d50 2024-04-23T18:19:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.362+0000 7f3a8dffb640 1 -- 172.21.15.37:0/3308796100 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98035070 con 0x555df0836d50 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3308796100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 msgr2=0x555df08371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df08371d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009230 tx=0x7f3a98032e70 comp rx=0 tx=0).stop 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3308796100 shutdown_connections 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df083dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df08371d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3308796100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df0835fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3308796100 >> 172.21.15.37:0/3308796100 conn(0x555df0831070 msgr2=0x555df0831470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3308796100 shutdown_connections 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3308796100 wait complete. 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.363+0000 7f3ab0dd6740 1 -- start start 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 Processor -- start 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df083ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df0840510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df0847550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555df083e6c0 con 0x555df0837710 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555df0735470 con 0x555df0835b80 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555df083e840 con 0x555df0836d50 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3ab0dd6740 10 client.0 _ll_register_callbacks cb 0x555df075af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df0847550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df0847550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58398/0 (socket says 172.21.15.37:58398) 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/4142350205 learned_addr learned my addr 172.21.15.37:0/4142350205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df0840510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 msgr2=0x555df0840510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df0840510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/4142350205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 msgr2=0x555df083ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df083ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/4142350205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df05776b0 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df0847550 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c005fb0 tx=0x7f3a9c012e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bc78d082515fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.364+0000 7f3a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3a86ffd640 1 -- 172.21.15.37:0/4142350205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c021060 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3a86ffd640 1 -- 172.21.15.37:0/4142350205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3a9c010d20 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a980317a0 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3a86ffd640 1 -- 172.21.15.37:0/4142350205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3a9c01c420 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 10 client.15060 fetch_fsmap learned FSMap version 22 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555df084a930 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3a86ffd640 1 -- 172.21.15.37:0/4142350205 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a980317a0 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3a86ffd640 1 -- 172.21.15.37:0/4142350205 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3a9c00f3c0 con 0x555df0837710 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 10 client.15060 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:19:59.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 20 client.15060 populate_metadata read hostname 'smithi037' 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df084cfa0 0x555df084e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 4 client.15060 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555df07a8670 con 0x555df084cfa0 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.365+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df084cfa0 0x555df084e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.366+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df084cfa0 0x555df084e410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64727abb9de71bfa server_cookie=e71d6235d7c3832a in_seq=0 out_seq=0 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.366+0000 7f3a86ffd640 10 client.15060 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3a86ffd640 1 -- 172.21.15.37:0/4142350205 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555df07a8670 con 0x555df084cfa0 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3a86ffd640 10 client.15060 handle_command_reply: tid=0 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 client.15060 shutdown 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 msgr2=0x555df0847550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df0847550 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c005fb0 tx=0x7f3a9c012e40 comp rx=0 tx=0).stop 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 shutdown_connections 2024-04-23T18:19:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df084cfa0 0x555df084e410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0837710 0x555df0847550 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0836d50 0x555df0840510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/4142350205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0835b80 0x555df083ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3a86ffd640 0 client.15060 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 >> 172.21.15.37:0/4142350205 conn(0x555df0831070 msgr2=0x555df08301d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 shutdown_connections 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/4142350205 wait complete. 2024-04-23T18:19:59.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.367+0000 7f3ab0dd6740 20 client.15060 trim_cache size 0 max 16384 2024-04-23T18:19:59.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:59 smithi037 ceph-mon[29404]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:19:59.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:59 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3276956268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:59 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1922225823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:19:59 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4288753665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 Processor -- start 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 -- start start 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0835b80 0x555df084d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 0x555df083deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0837710 0x555df082c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555df084c750 con 0x555df0836d50 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555df084ce20 con 0x555df0837710 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555df084d4f0 con 0x555df0835b80 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 0x555df083deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 0x555df083deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58412/0 (socket says 172.21.15.37:58412) 2024-04-23T18:19:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.385+0000 7f3aa77fe640 1 -- 172.21.15.37:0/2681872849 learned_addr learned my addr 172.21.15.37:0/2681872849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0837710 0x555df082c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa77fe640 1 -- 172.21.15.37:0/2681872849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0835b80 msgr2=0x555df084d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0835b80 0x555df084d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa77fe640 1 -- 172.21.15.37:0/2681872849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0837710 msgr2=0x555df082c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0837710 0x555df082c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa77fe640 1 -- 172.21.15.37:0/2681872849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df083db30 con 0x555df0836d50 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 0x555df083deb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009900 tx=0x7f3a9802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34681c55ec98ebea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3a8d7fa640 1 -- 172.21.15.37:0/2681872849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9803f040 con 0x555df0836d50 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3a8d7fa640 1 -- 172.21.15.37:0/2681872849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98005040 con 0x555df0836d50 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.386+0000 7f3a8d7fa640 1 -- 172.21.15.37:0/2681872849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98035040 con 0x555df0836d50 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/2681872849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 msgr2=0x555df083deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 0x555df083deb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009900 tx=0x7f3a9802a3f0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/2681872849 shutdown_connections 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0837710 0x555df082c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0836d50 0x555df083deb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/2681872849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0835b80 0x555df084d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/2681872849 >> 172.21.15.37:0/2681872849 conn(0x555df0831070 msgr2=0x555df0831450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/2681872849 shutdown_connections 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/2681872849 wait complete. 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- start start 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 Processor -- start 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 0x555df0846950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0836d50 0x555df0846e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0837710 0x555df08b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555df084e290 con 0x555df0835b80 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555df084dbc0 con 0x555df0836d50 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555df084e960 con 0x555df0837710 2024-04-23T18:19:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3ab0dd6740 10 client.0 _ll_register_callbacks cb 0x7f3a9c012660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 0x555df0846950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 0x555df0846950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58420/0 (socket says 172.21.15.37:58420) 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.387+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/3481338919 learned_addr learned my addr 172.21.15.37:0/3481338919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0837710 msgr2=0x555df08b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0836d50 0x555df0846e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0837710 0x555df08b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/3481338919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0836d50 msgr2=0x555df0846e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0836d50 0x555df0846e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/3481338919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df08b58a0 con 0x555df0835b80 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 0x555df0846950 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00f9d0 tx=0x7f3a9c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35a48ed6b6e16d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3a86ffd640 1 -- 172.21.15.37:0/3481338919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c021040 con 0x555df0835b80 2024-04-23T18:19:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3a86ffd640 1 -- 172.21.15.37:0/3481338919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f3a9c0273f0 con 0x555df0835b80 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a98005040 con 0x555df0835b80 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3a86ffd640 1 -- 172.21.15.37:0/3481338919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3a9c005750 con 0x555df0835b80 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3ab0dd6740 10 client.15066 fetch_fsmap learned FSMap version 22 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555df08b6bd0 con 0x555df0835b80 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.388+0000 7f3a86ffd640 1 -- 172.21.15.37:0/3481338919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a98005040 con 0x555df0835b80 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3a86ffd640 1 -- 172.21.15.37:0/3481338919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3a9c039050 con 0x555df0835b80 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3ab0dd6740 10 client.15066 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3ab0dd6740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3ab0dd6740 20 client.15066 populate_metadata read hostname 'smithi037' 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df08bb7d0 0x555df08ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3ab0dd6740 4 client.15066 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "status"}) -- 0x555df07a24e0 con 0x555df08bb7d0 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df08bb7d0 0x555df08ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df08bb7d0 0x555df08ba4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6914c29a7f970431 server_cookie=ecb695c2db6dd850 in_seq=0 out_seq=0 2024-04-23T18:19:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.389+0000 7f3a86ffd640 10 client.15066 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3a86ffd640 1 -- 172.21.15.37:0/3481338919 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555df07a24e0 con 0x555df08bb7d0 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3a86ffd640 10 client.15066 handle_command_reply: tid=0 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 client.15066 shutdown 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 msgr2=0x555df0846950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 0x555df0846950 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00f9d0 tx=0x7f3a9c00dd10 comp rx=0 tx=0).stop 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 shutdown_connections 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x555df08bb7d0 0x555df08ba4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555df0837710 0x555df08b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3a86ffd640 0 client.15066 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555df0836d50 0x555df0846e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 --2- 172.21.15.37:0/3481338919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555df0835b80 0x555df0846950 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 >> 172.21.15.37:0/3481338919 conn(0x555df0831070 msgr2=0x555df082c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 shutdown_connections 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 1 -- 172.21.15.37:0/3481338919 wait complete. 2024-04-23T18:19:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3ab0dd6740 20 client.15066 trim_cache size 0 max 16384 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24484, 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 81.324952468999996, 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-04-23T18:19:59.390 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-23T18:19:59.391 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:19:59.391 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 99.142324631999998 2024-04-23T18:19:59.391 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:19:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a900a3c40 msgr2=0x7f3a900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a900a3c40 0x7f3a900a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000efd0 tx=0x7f3aa0087040 comp rx=0 tx=0).stop 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 msgr2=0x7f3aa819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000eaa0 tx=0x7f3aa000ef60 comp rx=0 tx=0).stop 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 shutdown_connections 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f3a900a3c40 0x7f3a900a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa8107c70 0x7f3aa819f720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3aa8105340 0x7f3aa81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 --2- 172.21.15.37:0/736336794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aa8102a10 0x7f3aa8198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.390+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 >> 172.21.15.37:0/736336794 conn(0x7f3aa80fc400 msgr2=0x7f3aa8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.391+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 shutdown_connections 2024-04-23T18:19:59.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.391+0000 7f3aae0cb640 1 -- 172.21.15.37:0/736336794 wait complete. 2024-04-23T18:19:59.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:19:59.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 -- 172.21.15.37:0/3812695742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 msgr2=0x7f9980107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 --2- 172.21.15.37:0/3812695742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980107830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9968009900 tx=0x7f996802a410 comp rx=0 tx=0).stop 2024-04-23T18:19:59.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 -- 172.21.15.37:0/3812695742 shutdown_connections 2024-04-23T18:19:59.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 --2- 172.21.15.37:0/3812695742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9980107d70 0x7f998010a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 --2- 172.21.15.37:0/3812695742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980107830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 --2- 172.21.15.37:0/3812695742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9980102b10 0x7f9980104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 -- 172.21.15.37:0/3812695742 >> 172.21.15.37:0/3812695742 conn(0x7f99800fc4a0 msgr2=0x7f99800fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 -- 172.21.15.37:0/3812695742 shutdown_connections 2024-04-23T18:19:59.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 -- 172.21.15.37:0/3812695742 wait complete. 2024-04-23T18:19:59.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.505+0000 7f9984a35640 1 Processor -- start 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 -- start start 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9980102b10 0x7f9980102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9980107d70 0x7f9980101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f998010dbb0 con 0x7f9980105440 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f998010da30 con 0x7f9980102b10 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9984a35640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f998010dd30 con 0x7f9980107d70 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58438/0 (socket says 172.21.15.37:58438) 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 -- 172.21.15.37:0/545209023 learned_addr learned my addr 172.21.15.37:0/545209023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f997ed9d640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9980107d70 0x7f9980101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f997e59c640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9980102b10 0x7f9980102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 -- 172.21.15.37:0/545209023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9980107d70 msgr2=0x7f9980101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9980107d70 0x7f9980101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 -- 172.21.15.37:0/545209023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9980102b10 msgr2=0x7f9980102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9980102b10 0x7f9980102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 -- 172.21.15.37:0/545209023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9980101890 con 0x7f9980105440 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.506+0000 7f9975d9b640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980100ac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9968006010 tx=0x7f99680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.507+0000 7f9977fff640 1 -- 172.21.15.37:0/545209023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996803f030 con 0x7f9980105440 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.507+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99801a2380 con 0x7f9980105440 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.507+0000 7f9977fff640 1 -- 172.21.15.37:0/545209023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99680318f0 con 0x7f9980105440 2024-04-23T18:19:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.507+0000 7f9977fff640 1 -- 172.21.15.37:0/545209023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968035070 con 0x7f9980105440 2024-04-23T18:19:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.507+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99801a28d0 con 0x7f9980105440 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.509+0000 7f9977fff640 1 -- 172.21.15.37:0/545209023 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9968042030 con 0x7f9980105440 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.509+0000 7f998782f740 1 Processor -- start 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 -- start start 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e6dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e6f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e75fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b55dd2c50 con 0x560b55e6db80 2024-04-23T18:19:59.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b55d6f830 con 0x560b55e6f710 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f998782f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560b55dcec10 con 0x560b55e6ed50 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e6dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e6dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58442/0 (socket says 172.21.15.37:58442) 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 -- 172.21.15.37:0/1332226566 learned_addr learned my addr 172.21.15.37:0/1332226566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f9977fff640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f996c0a3c40 0x7f996c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f9975d9b640 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e75fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997e59c640 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e6f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f996c0a3c40 0x7f996c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 -- 172.21.15.37:0/1332226566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 msgr2=0x560b55e6f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e6f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 -- 172.21.15.37:0/1332226566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 msgr2=0x560b55e75fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e75fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f997ed9d640 1 -- 172.21.15.37:0/1332226566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b55ae52d0 con 0x560b55e6db80 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.510+0000 7f9977fff640 1 -- 172.21.15.37:0/545209023 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f99680f0b00 con 0x7f9980105440 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f997ed9d640 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e6dfa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9978005c40 tx=0x7f997800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a74658c359da37f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f9974d99640 1 -- 172.21.15.37:0/1332226566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978022020 con 0x560b55e6db80 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f9974d99640 1 -- 172.21.15.37:0/1332226566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9978013170 con 0x560b55e6db80 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f9974d99640 1 -- 172.21.15.37:0/1332226566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978010690 con 0x560b55e6db80 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 -- 172.21.15.37:0/1332226566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 msgr2=0x560b55e6dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e6dfa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9978005c40 tx=0x7f997800f3c0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f997ed9d640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f996c0a3c40 0x7f996c0a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9978023000 tx=0x7f9978021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 -- 172.21.15.37:0/1332226566 shutdown_connections 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e75fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e6f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 --2- 172.21.15.37:0/1332226566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e6dfa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.511+0000 7f998782f740 1 -- 172.21.15.37:0/1332226566 >> 172.21.15.37:0/1332226566 conn(0x560b55e69070 msgr2=0x560b55e69470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 -- 172.21.15.37:0/1332226566 shutdown_connections 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 -- 172.21.15.37:0/1332226566 wait complete. 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 -- start start 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 Processor -- start 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6db80 0x560b55e78060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e7f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b55e76720 con 0x560b55e6f710 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b55d6d470 con 0x560b55e6db80 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560b55e768a0 con 0x560b55e6ed50 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f998782f740 10 client.0 _ll_register_callbacks cb 0x560b55d92f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f997ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e7f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f997ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e7f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58454/0 (socket says 172.21.15.37:58454) 2024-04-23T18:19:59.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f997ed9d640 1 -- 172.21.15.37:0/2374216625 learned_addr learned my addr 172.21.15.37:0/2374216625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f997e59c640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6db80 0x560b55e78060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.512+0000 7f9975d9b640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997ed9d640 1 -- 172.21.15.37:0/2374216625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 msgr2=0x560b55e785a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997ed9d640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e785a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997ed9d640 1 -- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6db80 msgr2=0x560b55e78060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997ed9d640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6db80 0x560b55e78060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997ed9d640 1 -- 172.21.15.37:0/2374216625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b55baf6b0 con 0x560b55e6f710 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997e59c640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6db80 0x560b55e78060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f997ed9d640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e7f5e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f997800ffc0 tx=0x7f997800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ffe9720e94094 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f9965ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f9965ffb640 1 -- 172.21.15.37:0/2374216625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978034060 con 0x560b55e6f710 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9978013170 con 0x560b55e6f710 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f9965ffb640 1 -- 172.21.15.37:0/2374216625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9978010630 con 0x560b55e6f710 2024-04-23T18:19:59.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f9965ffb640 1 -- 172.21.15.37:0/2374216625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9978022070 con 0x560b55e6f710 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f998782f740 10 client.15078 fetch_fsmap learned FSMap version 22 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560b55e829c0 con 0x560b55e6f710 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f9965ffb640 1 -- 172.21.15.37:0/2374216625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9978013170 con 0x560b55e6f710 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.513+0000 7f9965ffb640 1 -- 172.21.15.37:0/2374216625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9978010970 con 0x560b55e6f710 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f998782f740 10 client.15078 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f998782f740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f998782f740 20 client.15078 populate_metadata read hostname 'smithi037' 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f998782f740 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55e84de0 0x560b55e86250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f998782f740 4 client.15078 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b55de0670 con 0x560b55e84de0 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f997e59c640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55e84de0 0x560b55e86250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f997e59c640 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55e84de0 0x560b55e86250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad2652aea99b060 server_cookie=49e838e2b23dab61 in_seq=0 out_seq=0 2024-04-23T18:19:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.514+0000 7f9965ffb640 10 client.15078 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f9965ffb640 1 -- 172.21.15.37:0/2374216625 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b55de0670 con 0x560b55e84de0 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f9965ffb640 10 client.15078 handle_command_reply: tid=0 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 client.15078 shutdown 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 msgr2=0x560b55e7f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e7f5e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f997800ffc0 tx=0x7f997800f970 comp rx=0 tx=0).stop 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 shutdown_connections 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55e84de0 0x560b55e86250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e7f5e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 --2- 172.21.15.37:0/2374216625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6db80 0x560b55e78060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 >> 172.21.15.37:0/2374216625 conn(0x560b55e69070 msgr2=0x560b55e681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f9965ffb640 0 client.15078 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 shutdown_connections 2024-04-23T18:19:59.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 1 -- 172.21.15.37:0/2374216625 wait complete. 2024-04-23T18:19:59.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.515+0000 7f998782f740 20 client.15078 trim_cache size 0 max 16384 2024-04-23T18:19:59.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.533+0000 7f998782f740 1 Processor -- start 2024-04-23T18:19:59.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.533+0000 7f998782f740 1 -- start start 2024-04-23T18:19:59.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.533+0000 7f998782f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6db80 0x560b55e78500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.533+0000 7f998782f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6ed50 0x560b55e7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.533+0000 7f998782f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e83dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f998782f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b55e84590 con 0x560b55e6f710 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f998782f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b55e84c60 con 0x560b55e6ed50 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f998782f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560b55e85330 con 0x560b55e6db80 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e83dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e83dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58456/0 (socket says 172.21.15.37:58456) 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 -- 172.21.15.37:0/3397900748 learned_addr learned my addr 172.21.15.37:0/3397900748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 -- 172.21.15.37:0/3397900748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6db80 msgr2=0x560b55e78500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6db80 0x560b55e78500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 -- 172.21.15.37:0/3397900748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6ed50 msgr2=0x560b55e7f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6ed50 0x560b55e7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 -- 172.21.15.37:0/3397900748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b55e6d640 con 0x560b55e6f710 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f997ed9d640 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e83dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9978003f90 tx=0x7f9978009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e1200f887c7e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f9967fff640 1 -- 172.21.15.37:0/3397900748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978034030 con 0x560b55e6f710 2024-04-23T18:19:59.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f9967fff640 1 -- 172.21.15.37:0/3397900748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9978005850 con 0x560b55e6f710 2024-04-23T18:19:59.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.534+0000 7f9967fff640 1 -- 172.21.15.37:0/3397900748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99780106b0 con 0x560b55e6f710 2024-04-23T18:19:59.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- 172.21.15.37:0/3397900748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 msgr2=0x560b55e83dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e83dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9978003f90 tx=0x7f9978009b10 comp rx=0 tx=0).stop 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- 172.21.15.37:0/3397900748 shutdown_connections 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6f710 0x560b55e83dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6ed50 0x560b55e7f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- 172.21.15.37:0/3397900748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6db80 0x560b55e78500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- 172.21.15.37:0/3397900748 >> 172.21.15.37:0/3397900748 conn(0x560b55e69070 msgr2=0x560b55e69450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- 172.21.15.37:0/3397900748 shutdown_connections 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- 172.21.15.37:0/3397900748 wait complete. 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- start start 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 Processor -- start 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e7b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e7da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b55e860d0 con 0x560b55e6db80 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b55e85a00 con 0x560b55e6f710 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560b55e867a0 con 0x560b55e6ed50 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f998782f740 10 client.0 _ll_register_callbacks cb 0x7f99680b87f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f997e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e7b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f997e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e7b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58470/0 (socket says 172.21.15.37:58470) 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.535+0000 7f997e59c640 1 -- 172.21.15.37:0/2078922580 learned_addr learned my addr 172.21.15.37:0/2078922580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f997e59c640 1 -- 172.21.15.37:0/2078922580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 msgr2=0x560b55e7da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f997e59c640 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e7da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f997e59c640 1 -- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 msgr2=0x560b55e7df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f997e59c640 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f997e59c640 1 -- 172.21.15.37:0/2078922580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b55e78f00 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f997e59c640 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e7b4e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9970006df0 tx=0x7f997001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d93ab80bd740d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f9965ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f9965ffb640 1 -- 172.21.15.37:0/2078922580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970016aa0 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9978005850 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f9965ffb640 1 -- 172.21.15.37:0/2078922580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f997000f040 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f9965ffb640 1 -- 172.21.15.37:0/2078922580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9970016c40 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f998782f740 10 client.15084 fetch_fsmap learned FSMap version 22 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560b55e7f640 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f9965ffb640 1 -- 172.21.15.37:0/2078922580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9978005850 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f9965ffb640 1 -- 172.21.15.37:0/2078922580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9970025660 con 0x560b55e6db80 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.536+0000 7f998782f740 10 client.15084 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 20 client.15084 populate_metadata read hostname 'smithi037' 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55eef450 0x560b55eee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:59.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 4 client.15084 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "status"}) -- 0x560b55dda4e0 con 0x560b55eef450 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f9975d9b640 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55eef450 0x560b55eee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f9965ffb640 10 client.15084 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f9975d9b640 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55eef450 0x560b55eee160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580b92f65f8dffaa server_cookie=7f417827496abcb2 in_seq=0 out_seq=0 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f9965ffb640 1 -- 172.21.15.37:0/2078922580 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x560b55dda4e0 con 0x560b55eef450 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f9965ffb640 10 client.15084 handle_command_reply: tid=0 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 1 client.15084 shutdown 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 msgr2=0x560b55e7b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.537+0000 7f998782f740 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e7b4e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9970006df0 tx=0x7f997001cf20 comp rx=0 tx=0).stop 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 shutdown_connections 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560b55eef450 0x560b55eee160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b55e6f710 0x560b55e7df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560b55e6ed50 0x560b55e7da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 --2- 172.21.15.37:0/2078922580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b55e6db80 0x560b55e7b4e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9965ffb640 0 client.15084 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 >> 172.21.15.37:0/2078922580 conn(0x560b55e69070 msgr2=0x560b55e83dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 shutdown_connections 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 1 -- 172.21.15.37:0/2078922580 wait complete. 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f998782f740 20 client.15084 trim_cache size 0 max 16384 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24509, 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:19:59.537 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:19:59.538 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 84.501472393, 2024-04-23T18:19:59.538 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-04-23T18:19:59.538 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-23T18:19:59.538 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:19:59.538 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 91.005691330999994 2024-04-23T18:19:59.538 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f996c0a3c40 msgr2=0x7f996c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f996c0a3c40 0x7f996c0a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9978023000 tx=0x7f9978021000 comp rx=0 tx=0).stop 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 msgr2=0x7f9980100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980100ac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9968006010 tx=0x7f99680042f0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 shutdown_connections 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f996c0a3c40 0x7f996c0a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9980107d70 0x7f9980101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9980105440 0x7f9980100ac0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 --2- 172.21.15.37:0/545209023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9980102b10 0x7f9980102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 >> 172.21.15.37:0/545209023 conn(0x7f99800fc4a0 msgr2=0x7f9980103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 shutdown_connections 2024-04-23T18:19:59.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:19:59.538+0000 7f9984a35640 1 -- 172.21.15.37:0/545209023 wait complete. 2024-04-23T18:19:59.542 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T18:19:59.543 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:19:59.555 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:19:59.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T18:19:59.801 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:00.194 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:00 smithi037 ceph-mon[29404]: overall HEALTH_OK 2024-04-23T18:20:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:00 smithi173 ceph-mon[32906]: overall HEALTH_OK 2024-04-23T18:20:00.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:00 smithi172 ceph-mon[32456]: overall HEALTH_OK 2024-04-23T18:20:01.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:01 smithi037 ceph-mon[29404]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:20:01.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 -- 172.21.15.37:0/2686888618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 msgr2=0x7f5eb0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 --2- 172.21.15.37:0/2686888618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb0101dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000b0d0 tx=0x7f5ea000b3e0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 -- 172.21.15.37:0/2686888618 shutdown_connections 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 --2- 172.21.15.37:0/2686888618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb0101dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 --2- 172.21.15.37:0/2686888618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 0x7f5eb0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 --2- 172.21.15.37:0/2686888618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eb0102ba0 0x7f5eb0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 -- 172.21.15.37:0/2686888618 >> 172.21.15.37:0/2686888618 conn(0x7f5eb00fcf40 msgr2=0x7f5eb00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 -- 172.21.15.37:0/2686888618 shutdown_connections 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.350+0000 7f5eb6164700 1 -- 172.21.15.37:0/2686888618 wait complete. 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 Processor -- start 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 -- start start 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb006b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eb0102ba0 0x7f5eb019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 0x7f5eb019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb010d410 con 0x7f5eb01019a0 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb010d290 con 0x7f5eb0103540 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eb6164700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb010d590 con 0x7f5eb0102ba0 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb006b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb006b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58482/0 (socket says 172.21.15.37:58482) 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eaf7fe700 1 -- 172.21.15.37:0/3472095164 learned_addr learned my addr 172.21.15.37:0/3472095164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eaeffd700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eb0102ba0 0x7f5eb019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.351+0000 7f5eaffff700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 0x7f5eb019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaf7fe700 1 -- 172.21.15.37:0/3472095164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eb0102ba0 msgr2=0x7f5eb019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaf7fe700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eb0102ba0 0x7f5eb019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaf7fe700 1 -- 172.21.15.37:0/3472095164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 msgr2=0x7f5eb019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaf7fe700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 0x7f5eb019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaf7fe700 1 -- 172.21.15.37:0/3472095164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb01a0880 con 0x7f5eb01019a0 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaffff700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 0x7f5eb019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eaf7fe700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb006b2a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000d010 tx=0x7f5ea0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0003c00 con 0x7f5eb01019a0 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb01a0b10 con 0x7f5eb01019a0 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ea0003da0 con 0x7f5eb01019a0 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea001b840 con 0x7f5eb01019a0 2024-04-23T18:20:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.352+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb01a0fc0 con 0x7f5eb01019a0 2024-04-23T18:20:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.353+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb0069730 con 0x7f5eb01019a0 2024-04-23T18:20:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.354+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ea0026450 con 0x7f5eb01019a0 2024-04-23T18:20:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.355+0000 7f5eacff9700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5e9c0a3cd0 0x7f5e9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.355+0000 7f5eaeffd700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5e9c0a3cd0 0x7f5e9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.355+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f5ea000f020 con 0x7f5eb01019a0 2024-04-23T18:20:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.355+0000 7f5eaeffd700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5e9c0a3cd0 0x7f5e9c0a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5eb006b800 tx=0x7f5e9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.360+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ea009e1b0 con 0x7f5eb01019a0 2024-04-23T18:20:01.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:01 smithi173 ceph-mon[32906]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:20:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.588+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5eb0062e80 con 0x7f5e9c0a3cd0 2024-04-23T18:20:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.590+0000 7f5eacff9700 1 -- 172.21.15.37:0/3472095164 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5eb0062e80 con 0x7f5e9c0a3cd0 2024-04-23T18:20:01.583 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-04-23T18:20:01.583 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-04-23T18:20:01.583 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-04-23T18:20:01.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5e9c0a3cd0 msgr2=0x7f5e9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:01.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5e9c0a3cd0 0x7f5e9c0a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5eb006b800 tx=0x7f5e9800b410 comp rx=0 tx=0).stop 2024-04-23T18:20:01.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 msgr2=0x7f5eb006b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb006b2a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000d010 tx=0x7f5ea0007a20 comp rx=0 tx=0).stop 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 shutdown_connections 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5e9c0a3cd0 0x7f5e9c0a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eb01019a0 0x7f5eb006b2a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0103540 0x7f5eb019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 --2- 172.21.15.37:0/3472095164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eb0102ba0 0x7f5eb019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 >> 172.21.15.37:0/3472095164 conn(0x7f5eb00fcf40 msgr2=0x7f5eb0105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 shutdown_connections 2024-04-23T18:20:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:01.592+0000 7f5eb6164700 1 -- 172.21.15.37:0/3472095164 wait complete. 2024-04-23T18:20:01.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:01 smithi172 ceph-mon[32456]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:20:02.188 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:02 smithi037 ceph-mon[29404]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:02.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T18:20:02.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:02 smithi173 ceph-mon[32906]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:02.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:02 smithi172 ceph-mon[32456]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:03.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:03 smithi037 ceph-mon[29404]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T18:20:03.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:03 smithi173 ceph-mon[32906]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T18:20:03.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:03 smithi172 ceph-mon[32456]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T18:20:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 -- 172.21.15.37:0/2063752488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa41035d0 msgr2=0x7feaa41059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:04.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2063752488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa41035d0 0x7feaa41059c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fea9400b0a0 tx=0x7fea9400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 -- 172.21.15.37:0/2063752488 shutdown_connections 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2063752488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa41035d0 0x7feaa41059c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2063752488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa4108830 0x7feaa410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2063752488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa4105f00 0x7feaa41082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 -- 172.21.15.37:0/2063752488 >> 172.21.15.37:0/2063752488 conn(0x7feaa40fcf40 msgr2=0x7feaa40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 -- 172.21.15.37:0/2063752488 shutdown_connections 2024-04-23T18:20:04.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 -- 172.21.15.37:0/2063752488 wait complete. 2024-04-23T18:20:04.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.095+0000 7feaa94ad700 1 Processor -- start 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 -- start start 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa41035d0 0x7feaa4196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa4105f00 0x7feaa4196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 0x7feaa419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410e5c0 con 0x7feaa4108830 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410e440 con 0x7feaa4105f00 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa94ad700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410e740 con 0x7feaa41035d0 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 0x7feaa419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 0x7feaa419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58500/0 (socket says 172.21.15.37:58500) 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 -- 172.21.15.37:0/2249632861 learned_addr learned my addr 172.21.15.37:0/2249632861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa27fc700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa4105f00 0x7feaa4196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa2ffd700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa41035d0 0x7feaa4196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 -- 172.21.15.37:0/2249632861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa41035d0 msgr2=0x7feaa4196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa41035d0 0x7feaa4196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 -- 172.21.15.37:0/2249632861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa4105f00 msgr2=0x7feaa4196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa4105f00 0x7feaa4196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 -- 172.21.15.37:0/2249632861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa419e740 con 0x7feaa4108830 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.096+0000 7feaa37fe700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 0x7feaa419e040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00b480 tx=0x7fea9c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.097+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c015020 con 0x7feaa4108830 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.097+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa419ea30 con 0x7feaa4108830 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.097+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea9c011ce0 con 0x7feaa4108830 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.097+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c010bf0 con 0x7feaa4108830 2024-04-23T18:20:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.097+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa419efa0 con 0x7feaa4108830 2024-04-23T18:20:04.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.098+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa4069730 con 0x7feaa4108830 2024-04-23T18:20:04.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.099+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea9c018020 con 0x7feaa4108830 2024-04-23T18:20:04.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.099+0000 7fea8bfff700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea8c0a3cd0 0x7fea8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:04.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.099+0000 7feaa2ffd700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea8c0a3cd0 0x7fea8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:04.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.100+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fea9c0cfb10 con 0x7feaa4108830 2024-04-23T18:20:04.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.100+0000 7feaa2ffd700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea8c0a3cd0 0x7fea8c0a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea94009fd0 tx=0x7fea9400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.104+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea9c01f020 con 0x7feaa4108830 2024-04-23T18:20:04.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.321+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7feaa4109f00 con 0x7fea8c0a3cd0 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.337+0000 7fea8bfff700 1 -- 172.21.15.37:0/2249632861 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7feaa4109f00 con 0x7fea8c0a3cd0 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (98s) 15s ago 98s 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 cb1eb13113c1 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi172 running (96s) 29s ago 95s 24.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7aabb44825f8 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi173 running (109s) 29s ago 109s 17.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 edc689ee73c2 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (117s) 15s ago 117s 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3fb45d89b531 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi172 running (114s) 29s ago 114s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f9baf81f3881 2024-04-23T18:20:04.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi173 running (112s) 29s ago 111s 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d3080db2a868 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (101s) 15s ago 101s 13.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a289d6c64445 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi172 running (107s) 29s ago 106s 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 96c8aaefce8d 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi173 running (104s) 29s ago 104s 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f395a5ca45a6 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (2m) 15s ago 119s 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f7505dc57dfa 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi172 running (2m) 29s ago 2m 14.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ca2429afcee7 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi173 running (2m) 29s ago 2m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ec75cb689fbf 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (9m) 15s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6e1ef653973d 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi172 *:8443,8765 running (7m) 29s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5716a8fb5a75 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi173 *:8443,8765 running (7m) 29s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3348114d907d 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (9m) 15s ago 9m 49.2M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 21630b3f612d 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi172 running (7m) 29s ago 7m 39.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 3ae5bb191b5d 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi173 running (7m) 29s ago 7m 38.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ce753cebb77b 2024-04-23T18:20:04.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (6m) 15s ago 6m 48.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 107101d73306 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi172 running (6m) 29s ago 6m 49.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 25e7b8fcd364 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi173 running (6m) 29s ago 6m 60.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ebbff91f9e63 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (5m) 15s ago 5m 46.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 509dbb5b676d 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi172 running (5m) 29s ago 5m 53.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 2b34095d97f5 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi173 running (4m) 29s ago 4m 57.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 97e1cbf6ac90 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (4m) 15s ago 4m 52.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 9086879a7bcd 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi172 running (4m) 29s ago 4m 54.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d5e9c9881eef 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi173 running (3m) 29s ago 3m 49.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b84a9ead4bfc 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (3m) 15s ago 3m 57.0M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 6866629c5bbd 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi172 running (2m) 29s ago 2m 54.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fe9d26136968 2024-04-23T18:20:04.332 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi173 running (2m) 29s ago 2m 46.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 4c97afb0e8fe 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea8c0a3cd0 msgr2=0x7fea8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea8c0a3cd0 0x7fea8c0a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea94009fd0 tx=0x7fea9400b7b0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 msgr2=0x7feaa419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 0x7feaa419e040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00b480 tx=0x7fea9c00b790 comp rx=0 tx=0).stop 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 shutdown_connections 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fea8c0a3cd0 0x7fea8c0a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaa4108830 0x7feaa419e040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa41035d0 0x7feaa4196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 --2- 172.21.15.37:0/2249632861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa4105f00 0x7feaa4196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 >> 172.21.15.37:0/2249632861 conn(0x7feaa40fcf40 msgr2=0x7feaa4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 shutdown_connections 2024-04-23T18:20:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:04.339+0000 7feaa94ad700 1 -- 172.21.15.37:0/2249632861 wait complete. 2024-04-23T18:20:04.906 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T18:20:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:05.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:05 smithi037 ceph-mon[29404]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:20:05.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:05 smithi037 ceph-mon[29404]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:05.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:05 smithi173 ceph-mon[32906]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:20:05.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:05 smithi173 ceph-mon[32906]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:05.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:05 smithi172 ceph-mon[32456]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:20:05.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:05 smithi172 ceph-mon[32456]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:06.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 -- 172.21.15.37:0/2038512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa718105ec0 msgr2=0x7fa7181082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:06.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2038512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa718105ec0 0x7fa7181082b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009990 tx=0x7fa70c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:20:06.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 -- 172.21.15.37:0/2038512065 shutdown_connections 2024-04-23T18:20:06.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2038512065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7181087f0 0x7fa71810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:06.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2038512065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa718103590 0x7fa718105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2038512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa718105ec0 0x7fa7181082b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 -- 172.21.15.37:0/2038512065 >> 172.21.15.37:0/2038512065 conn(0x7fa7180fcf40 msgr2=0x7fa7180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 -- 172.21.15.37:0/2038512065 shutdown_connections 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.868+0000 7fa71ded5700 1 -- 172.21.15.37:0/2038512065 wait complete. 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 Processor -- start 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 -- start start 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa718103590 0x7fa718198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa718105ec0 0x7fa718199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 0x7fa7181a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810e5a0 con 0x7fa7181087f0 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810e420 con 0x7fa718103590 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.869+0000 7fa71ded5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810e720 con 0x7fa718105ec0 2024-04-23T18:20:06.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 0x7fa7181a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 0x7fa7181a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55588/0 (socket says 172.21.15.37:55588) 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 -- 172.21.15.37:0/2446317989 learned_addr learned my addr 172.21.15.37:0/2446317989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa716ffd700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa718105ec0 0x7fa718199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa7177fe700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa718103590 0x7fa718198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 -- 172.21.15.37:0/2446317989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa718105ec0 msgr2=0x7fa718199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa718105ec0 0x7fa718199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 -- 172.21.15.37:0/2446317989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa718103590 msgr2=0x7fa718198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa718103590 0x7fa718198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 -- 172.21.15.37:0/2446317989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7181a0860 con 0x7fa7181087f0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa717fff700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 0x7fa7181a0190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b9e0 tx=0x7fa71000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71000d8e0 con 0x7fa7181087f0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7181a0b50 con 0x7fa7181087f0 2024-04-23T18:20:06.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa710015420 con 0x7fa7181087f0 2024-04-23T18:20:06.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.870+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710014630 con 0x7fa7181087f0 2024-04-23T18:20:06.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.871+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a1090 con 0x7fa7181087f0 2024-04-23T18:20:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.872+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718069730 con 0x7fa7181087f0 2024-04-23T18:20:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.873+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa7100155d0 con 0x7fa7181087f0 2024-04-23T18:20:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.874+0000 7fa714ff9700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7000a3cd0 0x7fa7000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.874+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fa7100d0340 con 0x7fa7181087f0 2024-04-23T18:20:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.874+0000 7fa7177fe700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7000a3cd0 0x7fa7000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:06.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.875+0000 7fa7177fe700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7000a3cd0 0x7fa7000a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa71819a0f0 tx=0x7fa70800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:06.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:06.878+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa71009cb50 con 0x7fa7181087f0 2024-04-23T18:20:07.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.097+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa718062e80 con 0x7fa7000a3cd0 2024-04-23T18:20:07.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.103+0000 7fa714ff9700 1 -- 172.21.15.37:0/2446317989 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa718062e80 con 0x7fa7000a3cd0 2024-04-23T18:20:07.096 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T18:20:07.096 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 32s ago 2m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi172=b;smithi172=e;smithi172=h;smithi172=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-23T18:20:07.096 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 32s ago 7m smithi037=x;smithi172=y;smithi173=z;count:3 2024-04-23T18:20:07.096 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 32s ago 7m smithi037:172.21.15.37=a;smithi172:172.21.15.172=b;smithi173:172.21.15.173=c;count:3 2024-04-23T18:20:07.096 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 32s ago - 2024-04-23T18:20:07.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7000a3cd0 msgr2=0x7fa7000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:07.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7000a3cd0 0x7fa7000a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa71819a0f0 tx=0x7fa70800b410 comp rx=0 tx=0).stop 2024-04-23T18:20:07.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 msgr2=0x7fa7181a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 0x7fa7181a0190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b9e0 tx=0x7fa71000bcf0 comp rx=0 tx=0).stop 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 shutdown_connections 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fa7000a3cd0 0x7fa7000a6190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7181087f0 0x7fa7181a0190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa718103590 0x7fa718198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 --2- 172.21.15.37:0/2446317989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa718105ec0 0x7fa718199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 >> 172.21.15.37:0/2446317989 conn(0x7fa7180fcf40 msgr2=0x7fa718103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 shutdown_connections 2024-04-23T18:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:07.106+0000 7fa71ded5700 1 -- 172.21.15.37:0/2446317989 wait complete. 2024-04-23T18:20:07.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:07 smithi173 ceph-mon[32906]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:20:07.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:07 smithi037 ceph-mon[29404]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:20:07.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:07 smithi172 ceph-mon[32456]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:20:07.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T18:20:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:08.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:08 smithi037 ceph-mon[29404]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:08.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:08 smithi172 ceph-mon[32456]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:08.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:08 smithi173 ceph-mon[32906]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:09.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:09 smithi037 ceph-mon[29404]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:20:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 -- 172.21.15.37:0/1207718901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 msgr2=0x7f3998108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 --2- 172.21.15.37:0/1207718901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f3998108250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3988009990 tx=0x7f3988009ca0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 -- 172.21.15.37:0/1207718901 shutdown_connections 2024-04-23T18:20:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 --2- 172.21.15.37:0/1207718901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3998108790 0x7f399810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 --2- 172.21.15.37:0/1207718901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3998103530 0x7f3998105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 --2- 172.21.15.37:0/1207718901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f3998108250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 -- 172.21.15.37:0/1207718901 >> 172.21.15.37:0/1207718901 conn(0x7f39980fcf40 msgr2=0x7f39980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 -- 172.21.15.37:0/1207718901 shutdown_connections 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.482+0000 7f39a06db700 1 -- 172.21.15.37:0/1207718901 wait complete. 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 Processor -- start 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 -- start start 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3998103530 0x7f3998102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f39981014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3998108790 0x7f3998101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f399810e5f0 con 0x7f3998105e60 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f399810e470 con 0x7f3998103530 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f39a06db700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f399810e770 con 0x7f3998108790 2024-04-23T18:20:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f39981014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f39981014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55610/0 (socket says 172.21.15.37:55610) 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 -- 172.21.15.37:0/3769012510 learned_addr learned my addr 172.21.15.37:0/3769012510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399e477700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3998103530 0x7f3998102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399ec78700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3998108790 0x7f3998101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 -- 172.21.15.37:0/3769012510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3998108790 msgr2=0x7f3998101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3998108790 0x7f3998101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 -- 172.21.15.37:0/3769012510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3998103530 msgr2=0x7f3998102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3998103530 0x7f3998102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.483+0000 7f399dc76700 1 -- 172.21.15.37:0/3769012510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39981022f0 con 0x7f3998105e60 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.484+0000 7f399dc76700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f39981014e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f39880038a0 tx=0x7f39880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.484+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988026070 con 0x7f3998105e60 2024-04-23T18:20:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.484+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3988004000 con 0x7f3998105e60 2024-04-23T18:20:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.484+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39981a2e30 con 0x7f3998105e60 2024-04-23T18:20:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.484+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398800fd60 con 0x7f3998105e60 2024-04-23T18:20:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.484+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39981a32a0 con 0x7f3998105e60 2024-04-23T18:20:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.485+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3998069730 con 0x7f3998105e60 2024-04-23T18:20:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.487+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3988029070 con 0x7f3998105e60 2024-04-23T18:20:09.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.488+0000 7f398f7fe700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f39840a3d50 0x7f39840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:09.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.488+0000 7f399e477700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f39840a3d50 0x7f39840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:09.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.488+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f39880d1420 con 0x7f3998105e60 2024-04-23T18:20:09.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.488+0000 7f399e477700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f39840a3d50 0x7f39840a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3994005950 tx=0x7f399400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:09.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.491+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3988030020 con 0x7f3998105e60 2024-04-23T18:20:09.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:09 smithi172 ceph-mon[32456]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:20:09.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:09 smithi173 ceph-mon[32906]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:20:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.706+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3998062e80 con 0x7f39840a3d50 2024-04-23T18:20:09.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.707+0000 7f398f7fe700 1 -- 172.21.15.37:0/3769012510 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3998062e80 con 0x7f39840a3d50 2024-04-23T18:20:09.700 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-04-23T18:20:09.700 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-04-23T18:20:09.700 INFO:teuthology.orchestra.run.smithi037.stdout:smithi172 172.21.15.172 2024-04-23T18:20:09.700 INFO:teuthology.orchestra.run.smithi037.stdout:smithi173 172.21.15.173 2024-04-23T18:20:09.701 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-04-23T18:20:09.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f39840a3d50 msgr2=0x7f39840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f39840a3d50 0x7f39840a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3994005950 tx=0x7f399400b410 comp rx=0 tx=0).stop 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 msgr2=0x7f39981014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f39981014e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f39880038a0 tx=0x7f39880047b0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 shutdown_connections 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f39840a3d50 0x7f39840a6210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3998108790 0x7f3998101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3998103530 0x7f3998102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 --2- 172.21.15.37:0/3769012510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3998105e60 0x7f39981014e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:09.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 >> 172.21.15.37:0/3769012510 conn(0x7f39980fcf40 msgr2=0x7f3998103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 shutdown_connections 2024-04-23T18:20:09.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:09.709+0000 7f39a06db700 1 -- 172.21.15.37:0/3769012510 wait complete. 2024-04-23T18:20:10.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T18:20:10.604 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:11.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:11 smithi037 ceph-mon[29404]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:11.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:11 smithi037 ceph-mon[29404]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:20:11.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:11 smithi037 ceph-mon[29404]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:20:11.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:11 smithi172 ceph-mon[32456]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:11.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:11 smithi172 ceph-mon[32456]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:20:11.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:11 smithi172 ceph-mon[32456]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:20:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:11 smithi173 ceph-mon[32906]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:11 smithi173 ceph-mon[32906]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T18:20:11.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:11 smithi173 ceph-mon[32906]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 -- 172.21.15.37:0/2528867865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 msgr2=0x7f1ecc101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/2528867865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc101de0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0099c0 tx=0x7f1ebc009cd0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 -- 172.21.15.37:0/2528867865 shutdown_connections 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/2528867865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc101de0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/2528867865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc103560 0x7f1ecc109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/2528867865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc102bc0 0x7f1ecc103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.149+0000 7f1ed23ee700 1 -- 172.21.15.37:0/2528867865 >> 172.21.15.37:0/2528867865 conn(0x7f1ecc0fcf60 msgr2=0x7f1ecc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 -- 172.21.15.37:0/2528867865 shutdown_connections 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 -- 172.21.15.37:0/2528867865 wait complete. 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 Processor -- start 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 -- start start 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc105950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc102bc0 0x7f1ecc103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.150+0000 7f1ed23ee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc103560 0x7f1ecc1044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ed23ee700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10d480 con 0x7f1ecc1019c0 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ed23ee700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10d300 con 0x7f1ecc102bc0 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ed23ee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10d600 con 0x7f1ecc103560 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55616/0 (socket says 172.21.15.37:55616) 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 -- 172.21.15.37:0/1493844927 learned_addr learned my addr 172.21.15.37:0/1493844927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecb7fe700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc102bc0 0x7f1ecc103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ed098b700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc103560 0x7f1ecc1044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 -- 172.21.15.37:0/1493844927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc103560 msgr2=0x7f1ecc1044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc103560 0x7f1ecc1044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 -- 172.21.15.37:0/1493844927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc102bc0 msgr2=0x7f1ecc103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc102bc0 0x7f1ecc103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 -- 172.21.15.37:0/1493844927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc104d90 con 0x7f1ecc1019c0 2024-04-23T18:20:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ecbfff700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc105950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009f50 tx=0x7f1ebc0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc027070 con 0x7f1ecc1019c0 2024-04-23T18:20:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0040e0 con 0x7f1ecc1019c0 2024-04-23T18:20:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00f460 con 0x7f1ecc1019c0 2024-04-23T18:20:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1a2cf0 con 0x7f1ecc1019c0 2024-04-23T18:20:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.151+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a3140 con 0x7f1ecc1019c0 2024-04-23T18:20:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.153+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc069760 con 0x7f1ecc1019c0 2024-04-23T18:20:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.154+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1ebc02a070 con 0x7f1ecc1019c0 2024-04-23T18:20:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.154+0000 7f1ec97fa700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f1eb80a3cc0 0x7f1eb80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.154+0000 7f1ecb7fe700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f1eb80a3cc0 0x7f1eb80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.154+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f1ebc0d12f0 con 0x7f1ecc1019c0 2024-04-23T18:20:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.155+0000 7f1ecb7fe700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f1eb80a3cc0 0x7f1eb80a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400ac50 tx=0x7f1eb400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:12.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.161+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ebc05eff0 con 0x7f1ecc1019c0 2024-04-23T18:20:12.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.378+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1ecc0ff100 con 0x7f1eb80a3cc0 2024-04-23T18:20:12.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.380+0000 7f1ec97fa700 1 -- 172.21.15.37:0/1493844927 <== mgr.14150 v2:172.21.15.37:6800/3159991179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1ecc0ff100 con 0x7f1eb80a3cc0 2024-04-23T18:20:12.373 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T18:20:12.373 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T18:20:12.373 INFO:teuthology.orchestra.run.smithi037.stdout:smithi172 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:20:12.373 INFO:teuthology.orchestra.run.smithi037.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:20:12.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f1eb80a3cc0 msgr2=0x7f1eb80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:12.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f1eb80a3cc0 0x7f1eb80a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400ac50 tx=0x7f1eb400a380 comp rx=0 tx=0).stop 2024-04-23T18:20:12.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 msgr2=0x7f1ecc105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc105950 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009f50 tx=0x7f1ebc0057a0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 shutdown_connections 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f1eb80a3cc0 0x7f1eb80a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ecc1019c0 0x7f1ecc105950 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc103560 0x7f1ecc1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 --2- 172.21.15.37:0/1493844927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc102bc0 0x7f1ecc103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 >> 172.21.15.37:0/1493844927 conn(0x7f1ecc0fcf60 msgr2=0x7f1ecc0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 shutdown_connections 2024-04-23T18:20:12.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:12.382+0000 7f1ed23ee700 1 -- 172.21.15.37:0/1493844927 wait complete. 2024-04-23T18:20:12.996 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:20:13.006 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:20:13.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T18:20:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:13.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:13 smithi037 ceph-mon[29404]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T18:20:13.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:13 smithi037 ceph-mon[29404]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:13.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:13 smithi172 ceph-mon[32456]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T18:20:13.590 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:13 smithi172 ceph-mon[32456]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:13.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:13 smithi173 ceph-mon[32906]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T18:20:13.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:13 smithi173 ceph-mon[32906]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 -- 172.21.15.37:0/636576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16301025c0 msgr2=0x7f1630109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 --2- 172.21.15.37:0/636576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16301025c0 0x7f1630109e60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f162800b0a0 tx=0x7f162800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:20:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 -- 172.21.15.37:0/636576 shutdown_connections 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 --2- 172.21.15.37:0/636576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16301025c0 0x7f1630109e60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 --2- 172.21.15.37:0/636576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 0x7f16301033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 --2- 172.21.15.37:0/636576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1630101c00 0x7f1630102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 -- 172.21.15.37:0/636576 >> 172.21.15.37:0/636576 conn(0x7f16300fc6d0 msgr2=0x7f16300feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 -- 172.21.15.37:0/636576 shutdown_connections 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.841+0000 7f16370ff700 1 -- 172.21.15.37:0/636576 wait complete. 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 Processor -- start 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 -- start start 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 0x7f1630101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16301025c0 0x7f163019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 0x7f163019fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f163010d470 con 0x7f1630101c00 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f163010d2f0 con 0x7f1630102fa0 2024-04-23T18:20:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f16370ff700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f163010d5f0 con 0x7f16301025c0 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f1634e9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 0x7f1630101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f1634e9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 0x7f1630101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47482/0 (socket says 172.21.15.37:47482) 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f1634e9b700 1 -- 172.21.15.37:0/3034681583 learned_addr learned my addr 172.21.15.37:0/3034681583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f163569c700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 0x7f163019fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.842+0000 7f162ffff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16301025c0 0x7f163019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f1634e9b700 1 -- 172.21.15.37:0/3034681583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16301025c0 msgr2=0x7f163019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f1634e9b700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16301025c0 0x7f163019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f1634e9b700 1 -- 172.21.15.37:0/3034681583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 msgr2=0x7f163019fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f1634e9b700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 0x7f163019fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f1634e9b700 1 -- 172.21.15.37:0/3034681583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301a0770 con 0x7f1630101c00 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f163569c700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 0x7f163019fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f1634e9b700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 0x7f1630101650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f162400c990 tx=0x7f162400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624007b00 con 0x7f1630101c00 2024-04-23T18:20:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16301a0a60 con 0x7f1630101c00 2024-04-23T18:20:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1624007ca0 con 0x7f1630101c00 2024-04-23T18:20:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624018660 con 0x7f1630101c00 2024-04-23T18:20:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.843+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301a1070 con 0x7f1630101c00 2024-04-23T18:20:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.844+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1630069730 con 0x7f1630101c00 2024-04-23T18:20:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.845+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1624004030 con 0x7f1630101c00 2024-04-23T18:20:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.846+0000 7f162dffb700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16200a3c40 0x7f16200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.846+0000 7f162ffff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16200a3c40 0x7f16200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.846+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f16240d0460 con 0x7f1630101c00 2024-04-23T18:20:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.847+0000 7f162ffff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16200a3c40 0x7f16200a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f161c006fd0 tx=0x7f161c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:14.850+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1624014030 con 0x7f1630101c00 2024-04-23T18:20:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.105+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f163002e000 con 0x7f1630101c00 2024-04-23T18:20:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.105+0000 7f162dffb700 1 -- 172.21.15.37:0/3034681583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) ==== 76+0+3052 (secure 0 0 0) 0x7f162409cec0 con 0x7f1630101c00 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:e22 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 22 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-23T18:18:34.408115+0000 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-23T18:18:40.084107+0000 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-04-23T18:20:15.099 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 3 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24509,1=24484,2=24479} 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-04-23T18:20:15.100 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24509 members: 24479,24484,24509 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{0:24509} state up:active seq 3 addr [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.173:6836/3035435067,v1:172.21.15.173:6837/3035435067] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{1:24484} state up:active seq 6 addr [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{1:24454} state up:standby-replay seq 1 addr [v2:172.21.15.173:6834/2419231317,v1:172.21.15.173:6835/2419231317] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{2:24479} state up:active seq 7 addr [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{2:24446} state up:standby-replay seq 1 addr [v2:172.21.15.172:6834/2640036963,v1:172.21.15.172:6835/2640036963] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:15.101 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.37:6834/3749554731,v1:172.21.15.37:6835/3749554731] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.102 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.37:6836/3961977132,v1:172.21.15.37:6837/3961977132] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.102 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:14604} state up:standby seq 1 addr [v2:172.21.15.37:6838/514853653,v1:172.21.15.37:6839/514853653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.102 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{-1:14616} state up:standby seq 1 addr [v2:172.21.15.37:6840/2155547418,v1:172.21.15.37:6841/2155547418] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.102 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.172:6832/3950527293,v1:172.21.15.172:6833/3950527293] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.102 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.173:6832/2179419272,v1:172.21.15.173:6833/2179419272] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.107+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16200a3c40 msgr2=0x7f16200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.107+0000 7f16370ff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16200a3c40 0x7f16200a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f161c006fd0 tx=0x7f161c008040 comp rx=0 tx=0).stop 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.107+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 msgr2=0x7f1630101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.107+0000 7f16370ff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 0x7f1630101650 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f162400c990 tx=0x7f162400cd50 comp rx=0 tx=0).stop 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 shutdown_connections 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f16200a3c40 0x7f16200a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1630101c00 0x7f1630101650 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:15.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630102fa0 0x7f163019fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:15.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 --2- 172.21.15.37:0/3034681583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16301025c0 0x7f163019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:15.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 >> 172.21.15.37:0/3034681583 conn(0x7f16300fc6d0 msgr2=0x7f1630105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:15.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 shutdown_connections 2024-04-23T18:20:15.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:15.108+0000 7f16370ff700 1 -- 172.21.15.37:0/3034681583 wait complete. 2024-04-23T18:20:15.104 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:20:15.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:15 smithi037 ceph-mon[29404]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:20:15.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:15 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3034681583' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:20:15.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:15 smithi172 ceph-mon[32456]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:20:15.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:15 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3034681583' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:20:15.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:15 smithi173 ceph-mon[32906]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-23T18:20:15.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:15 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3034681583' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:20:15.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af48a038-019c-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-23T18:20:15.910 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:17.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:17 smithi037 ceph-mon[29404]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.428+0000 7feca2b00700 1 -- 172.21.15.37:0/460814167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 msgr2=0x7fec9c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.428+0000 7feca2b00700 1 --2- 172.21.15.37:0/460814167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c102ee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fec900099c0 tx=0x7fec90009cd0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- 172.21.15.37:0/460814167 shutdown_connections 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 --2- 172.21.15.37:0/460814167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c102ee0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 --2- 172.21.15.37:0/460814167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 0x7fec9c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 --2- 172.21.15.37:0/460814167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c103cc0 0x7fec9c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- 172.21.15.37:0/460814167 >> 172.21.15.37:0/460814167 conn(0x7fec9c0fe060 msgr2=0x7fec9c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- 172.21.15.37:0/460814167 shutdown_connections 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- 172.21.15.37:0/460814167 wait complete. 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 Processor -- start 2024-04-23T18:20:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- start start 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c103cc0 0x7fec9c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 0x7fec9c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10e530 con 0x7fec9c102ac0 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10e3b0 con 0x7fec9c103cc0 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.429+0000 7feca2b00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10e6b0 con 0x7fec9c104660 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47510/0 (socket says 172.21.15.37:47510) 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 -- 172.21.15.37:0/1801381433 learned_addr learned my addr 172.21.15.37:0/1801381433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7fec9bfff700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c103cc0 0x7fec9c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca109d700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 0x7fec9c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 -- 172.21.15.37:0/1801381433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 msgr2=0x7fec9c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 0x7fec9c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 -- 172.21.15.37:0/1801381433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c103cc0 msgr2=0x7fec9c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c103cc0 0x7fec9c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 -- 172.21.15.37:0/1801381433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c075c20 con 0x7fec9c102ac0 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca109d700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 0x7fec9c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca089c700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c0767e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec90009f50 tx=0x7fec900057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90027070 con 0x7fec9c102ac0 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90004160 con 0x7fec9c102ac0 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c1a2cd0 con 0x7fec9c102ac0 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.430+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90022d70 con 0x7fec9c102ac0 2024-04-23T18:20:17.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.431+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c1a3180 con 0x7fec9c102ac0 2024-04-23T18:20:17.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.432+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9c069730 con 0x7fec9c102ac0 2024-04-23T18:20:17.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.433+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fec9002a070 con 0x7fec9c102ac0 2024-04-23T18:20:17.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.433+0000 7fec99ffb700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec8c0a3c40 0x7fec8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:17.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.434+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fec900d1a80 con 0x7fec9c102ac0 2024-04-23T18:20:17.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.434+0000 7fec9bfff700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec8c0a3c40 0x7fec8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:17.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.434+0000 7fec9bfff700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec8c0a3c40 0x7fec8c0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec88005950 tx=0x7fec8800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:17.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.437+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec90031020 con 0x7fec9c102ac0 2024-04-23T18:20:17.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:17 smithi172 ceph-mon[32456]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T18:20:17.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:17 smithi173 ceph-mon[32906]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T18:20:17.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.644+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fec9c063650 con 0x7fec9c102ac0 2024-04-23T18:20:17.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.645+0000 7fec99ffb700 1 -- 172.21.15.37:0/1801381433 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5320 (secure 0 0 0) 0x7fec90013070 con 0x7fec9c102ac0 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 90 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:fsid af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-23T18:10:40.839640+0000 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-23T18:19:41.678555+0000 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-04-23T18:20:17.639 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/3727253948,v1:172.21.15.37:6803/3727253948] [v2:172.21.15.37:6804/3727253948,v1:172.21.15.37:6805/3727253948] exists,up d65e5f77-8366-4dbf-92bf-a34c53c514c5 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6800/1638984815,v1:172.21.15.172:6801/1638984815] [v2:172.21.15.172:6802/1638984815,v1:172.21.15.172:6803/1638984815] exists,up 28317ec5-ba88-452e-95ba-a312941588bc 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/4145857941,v1:172.21.15.173:6801/4145857941] [v2:172.21.15.173:6802/4145857941,v1:172.21.15.173:6803/4145857941] exists,up 00f910a1-90d1-485c-a253-d0028ce0cc8d 2024-04-23T18:20:17.640 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 25 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/1815509049,v1:172.21.15.37:6811/1815509049] [v2:172.21.15.37:6812/1815509049,v1:172.21.15.37:6813/1815509049] exists,up 6533c224-ca86-4499-9182-283a32bb2774 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 33 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6808/3887503569,v1:172.21.15.172:6809/3887503569] [v2:172.21.15.172:6810/3887503569,v1:172.21.15.172:6811/3887503569] exists,up 1e24984b-edbd-4d4f-8716-6713ce2a039c 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/1194532318,v1:172.21.15.173:6809/1194532318] [v2:172.21.15.173:6810/1194532318,v1:172.21.15.173:6811/1194532318] exists,up c0152f97-2416-42e9-b0a0-6a7389cb0c02 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/825947067,v1:172.21.15.37:6819/825947067] [v2:172.21.15.37:6820/825947067,v1:172.21.15.37:6821/825947067] exists,up 5bc899de-4661-4c4f-b990-07db5cf3bd6e 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 53 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6816/3090366504,v1:172.21.15.172:6817/3090366504] [v2:172.21.15.172:6818/3090366504,v1:172.21.15.172:6819/3090366504] exists,up 1fb4b1b9-9fb4-475f-a40c-79ff6ba58324 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 61 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/1969719555,v1:172.21.15.173:6817/1969719555] [v2:172.21.15.173:6818/1969719555,v1:172.21.15.173:6819/1969719555] exists,up ec2a2669-fc68-4265-b68b-be4f008e4c2c 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 69 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/1483571052,v1:172.21.15.37:6827/1483571052] [v2:172.21.15.37:6828/1483571052,v1:172.21.15.37:6829/1483571052] exists,up 7eddb6e9-0b34-4a18-961f-0134670a2581 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 75 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6824/1398661350,v1:172.21.15.172:6825/1398661350] [v2:172.21.15.172:6826/1398661350,v1:172.21.15.172:6827/1398661350] exists,up 72e96128-329b-4155-83cf-58bb44169b73 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/1559500794,v1:172.21.15.173:6825/1559500794] [v2:172.21.15.173:6826/1559500794,v1:172.21.15.173:6827/1559500794] exists,up 3d918711-72d6-4a04-97cc-4fe8dad4a3e8 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.13 [2,11] 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.14 [2,4] 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1a [8,4] 2024-04-23T18:20:17.641 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.2 [5,6] 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.3 [4,9] 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.c [3,9,5,8] 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.14 [4,7] 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.1b [4,8] 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.3c [4,9] 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3125793004 expires 2024-04-24T18:11:39.779470+0000 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/4000629980 expires 2024-04-24T18:11:39.779470+0000 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3421057704 expires 2024-04-24T18:11:39.779470+0000 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1821395295 expires 2024-04-24T18:11:13.315493+0000 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1257180359 expires 2024-04-24T18:11:13.315493+0000 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/4000629980 expires 2024-04-24T18:11:39.779470+0000 2024-04-23T18:20:17.642 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/78121481 expires 2024-04-24T18:11:13.315493+0000 2024-04-23T18:20:17.643 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/1641631132 expires 2024-04-24T18:11:13.315493+0000 2024-04-23T18:20:17.643 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/1641631132 expires 2024-04-24T18:11:13.315493+0000 2024-04-23T18:20:17.643 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/4219584537 expires 2024-04-24T18:11:39.779470+0000 2024-04-23T18:20:17.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.647+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec8c0a3c40 msgr2=0x7fec8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:17.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.647+0000 7feca2b00700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec8c0a3c40 0x7fec8c0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec88005950 tx=0x7fec8800b410 comp rx=0 tx=0).stop 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.647+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 msgr2=0x7fec9c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.647+0000 7feca2b00700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c0767e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec90009f50 tx=0x7fec900057a0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 shutdown_connections 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fec8c0a3c40 0x7fec8c0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9c102ac0 0x7fec9c0767e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec9c104660 0x7fec9c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 --2- 172.21.15.37:0/1801381433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec9c103cc0 0x7fec9c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 >> 172.21.15.37:0/1801381433 conn(0x7fec9c0fe060 msgr2=0x7fec9c107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 shutdown_connections 2024-04-23T18:20:17.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:17.648+0000 7feca2b00700 1 -- 172.21.15.37:0/1801381433 wait complete. 2024-04-23T18:20:18.347 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:18 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1801381433' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:20:18.379 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-23T18:20:18.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:18.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 -- 172.21.15.37:0/2190256481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 msgr2=0x7f48681012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 --2- 172.21.15.37:0/2190256481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f48681012b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4858009900 tx=0x7f485802a3f0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 -- 172.21.15.37:0/2190256481 shutdown_connections 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 --2- 172.21.15.37:0/2190256481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48681029a0 0x7f48681092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 --2- 172.21.15.37:0/2190256481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101fe0 0x7f4868102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 --2- 172.21.15.37:0/2190256481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f48681012b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 -- 172.21.15.37:0/2190256481 >> 172.21.15.37:0/2190256481 conn(0x7f48680fc400 msgr2=0x7f48680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 -- 172.21.15.37:0/2190256481 shutdown_connections 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.504+0000 7f486d2a0640 1 -- 172.21.15.37:0/2190256481 wait complete. 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 Processor -- start 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 -- start start 2024-04-23T18:20:18.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f4868069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868101fe0 0x7f486819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48681029a0 0x7f486819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f486810c9c0 con 0x7f4868100e90 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f486810c840 con 0x7f48681029a0 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f486d2a0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f486810cb40 con 0x7f4868101fe0 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.505+0000 7f4866ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f4868069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f4868069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47532/0 (socket says 172.21.15.37:47532) 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 -- 172.21.15.37:0/3611390694 learned_addr learned my addr 172.21.15.37:0/3611390694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f48677fe640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48681029a0 0x7f486819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f48667fc640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868101fe0 0x7f486819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 -- 172.21.15.37:0/3611390694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868101fe0 msgr2=0x7f486819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868101fe0 0x7f486819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 -- 172.21.15.37:0/3611390694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48681029a0 msgr2=0x7f486819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48681029a0 0x7f486819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 -- 172.21.15.37:0/3611390694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486819fcf0 con 0x7f4868100e90 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f48667fc640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868101fe0 0x7f486819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f4866ffd640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f4868069cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4858031500 tx=0x7f48580043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485803f030 con 0x7f4868100e90 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4858003800 con 0x7f4868100e90 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486819ff80 con 0x7f4868100e90 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858035070 con 0x7f4868100e90 2024-04-23T18:20:18.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.506+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48681a0390 con 0x7f4868100e90 2024-04-23T18:20:18.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.507+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4868108140 con 0x7f4868100e90 2024-04-23T18:20:18.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.508+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f48580039b0 con 0x7f4868100e90 2024-04-23T18:20:18.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.509+0000 7f484ffff640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f48500a3bb0 0x7f48500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.509+0000 7f48667fc640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f48500a3bb0 0x7f48500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.509+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f48580f0ee0 con 0x7f4868100e90 2024-04-23T18:20:18.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.509+0000 7f48667fc640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f48500a3bb0 0x7f48500a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4868069a20 tx=0x7f485c005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:18.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.513+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48580bd8b0 con 0x7f4868100e90 2024-04-23T18:20:18.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:18 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1801381433' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:20:18.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:18 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1801381433' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:20:18.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.699+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4868062c90 con 0x7f4868100e90 2024-04-23T18:20:18.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.701+0000 7f484ffff640 1 -- 172.21.15.37:0/3611390694 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22684 (secure 0 0 0) 0x7f48580bd1b0 con 0x7f4868100e90 2024-04-23T18:20:18.695 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:18.695 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:19:41.678555+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f48500a3bb0 msgr2=0x7f48500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f48500a3bb0 0x7f48500a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4868069a20 tx=0x7f485c005db0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 msgr2=0x7f4868069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f4868069cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4858031500 tx=0x7f48580043c0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 shutdown_connections 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f48500a3bb0 0x7f48500a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48681029a0 0x7f486819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868101fe0 0x7f486819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 --2- 172.21.15.37:0/3611390694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4868100e90 0x7f4868069cb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 >> 172.21.15.37:0/3611390694 conn(0x7f48680fc400 msgr2=0x7f4868105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 shutdown_connections 2024-04-23T18:20:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.703+0000 7f486d2a0640 1 -- 172.21.15.37:0/3611390694 wait complete. 2024-04-23T18:20:18.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:20:18.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:20:18.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 -- 172.21.15.37:0/274721477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 msgr2=0x7fc41c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 --2- 172.21.15.37:0/274721477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c102460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc410009900 tx=0x7fc41002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 -- 172.21.15.37:0/274721477 shutdown_connections 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 --2- 172.21.15.37:0/274721477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc41c1029a0 0x7fc41c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 --2- 172.21.15.37:0/274721477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c102460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 --2- 172.21.15.37:0/274721477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 0x7fc41c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 -- 172.21.15.37:0/274721477 >> 172.21.15.37:0/274721477 conn(0x7fc41c0fc400 msgr2=0x7fc41c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 -- 172.21.15.37:0/274721477 shutdown_connections 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.819+0000 7fc421f20640 1 -- 172.21.15.37:0/274721477 wait complete. 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 Processor -- start 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 -- start start 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 0x7fc41c10b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc41c1029a0 0x7fc41c10a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c10db50 con 0x7fc41c101fe0 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c10d9d0 con 0x7fc41c100e90 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc421f20640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c10dcd0 con 0x7fc41c1029a0 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc41affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc41affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47548/0 (socket says 172.21.15.37:47548) 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.820+0000 7fc41affd640 1 -- 172.21.15.37:0/2881992878 learned_addr learned my addr 172.21.15.37:0/2881992878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41b7fe640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 0x7fc41c10b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41bfff640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc41c1029a0 0x7fc41c10a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41affd640 1 -- 172.21.15.37:0/2881992878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc41c1029a0 msgr2=0x7fc41c10a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41affd640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc41c1029a0 0x7fc41c10a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41affd640 1 -- 172.21.15.37:0/2881992878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 msgr2=0x7fc41c10b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41affd640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 0x7fc41c10b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41affd640 1 -- 172.21.15.37:0/2881992878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c10ac40 con 0x7fc41c101fe0 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41b7fe640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 0x7fc41c10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc41affd640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c109e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc410006010 tx=0x7fc410004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41003f030 con 0x7fc41c101fe0 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc410003cb0 con 0x7fc41c101fe0 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c1a2340 con 0x7fc41c101fe0 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410035070 con 0x7fc41c101fe0 2024-04-23T18:20:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.821+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c1a2910 con 0x7fc41c101fe0 2024-04-23T18:20:18.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.822+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41c108270 con 0x7fc41c101fe0 2024-04-23T18:20:18.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.823+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc410042030 con 0x7fc41c101fe0 2024-04-23T18:20:18.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.824+0000 7fc418ff9640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4040a3bb0 0x7fc4040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:18.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.824+0000 7fc41b7fe640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4040a3bb0 0x7fc4040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:18.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.824+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fc4100f0bd0 con 0x7fc41c101fe0 2024-04-23T18:20:18.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.824+0000 7fc41b7fe640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4040a3bb0 0x7fc4040a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc40c005c40 tx=0x7fc40c00a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:18.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:18.829+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc410003e60 con 0x7fc41c101fe0 2024-04-23T18:20:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.013+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc41c062b50 con 0x7fc41c101fe0 2024-04-23T18:20:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.014+0000 7fc418ff9640 1 -- 172.21.15.37:0/2881992878 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fc4100bd5a0 con 0x7fc41c101fe0 2024-04-23T18:20:19.008 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4040a3bb0 msgr2=0x7fc4040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4040a3bb0 0x7fc4040a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc40c005c40 tx=0x7fc40c00a430 comp rx=0 tx=0).stop 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 msgr2=0x7fc41c109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c109e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc410006010 tx=0x7fc410004430 comp rx=0 tx=0).stop 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 shutdown_connections 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fc4040a3bb0 0x7fc4040a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc41c1029a0 0x7fc41c10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc41c101fe0 0x7fc41c109e70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 --2- 172.21.15.37:0/2881992878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100e90 0x7fc41c10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 >> 172.21.15.37:0/2881992878 conn(0x7fc41c0fc400 msgr2=0x7fc41c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 shutdown_connections 2024-04-23T18:20:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.016+0000 7fc421f20640 1 -- 172.21.15.37:0/2881992878 wait complete. 2024-04-23T18:20:19.021 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 -- 172.21.15.37:0/1795256319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0103170 msgr2=0x7f9bd01035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 --2- 172.21.15.37:0/1795256319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0103170 0x7f9bd01035f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009900 tx=0x7f9bc402a410 comp rx=0 tx=0).stop 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 -- 172.21.15.37:0/1795256319 shutdown_connections 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 --2- 172.21.15.37:0/1795256319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd0103b30 0x7f9bd010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 --2- 172.21.15.37:0/1795256319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0103170 0x7f9bd01035f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 --2- 172.21.15.37:0/1795256319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0101f80 0x7f9bd01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 -- 172.21.15.37:0/1795256319 >> 172.21.15.37:0/1795256319 conn(0x7f9bd00fd510 msgr2=0x7f9bd00ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 -- 172.21.15.37:0/1795256319 shutdown_connections 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 -- 172.21.15.37:0/1795256319 wait complete. 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 Processor -- start 2024-04-23T18:20:19.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.135+0000 7f9bd8205640 1 -- start start 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 0x7f9bd01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd0103170 0x7f9bd01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0103b30 0x7f9bd019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010dac0 con 0x7f9bd0101f80 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010d940 con 0x7f9bd0103b30 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010dc40 con 0x7f9bd0103170 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 0x7f9bd01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 0x7f9bd01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47570/0 (socket says 172.21.15.37:47570) 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 -- 172.21.15.37:0/2306071860 learned_addr learned my addr 172.21.15.37:0/2306071860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd67a2640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0103b30 0x7f9bd019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 -- 172.21.15.37:0/2306071860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd0103170 msgr2=0x7f9bd01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd0103170 0x7f9bd01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 -- 172.21.15.37:0/2306071860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0103b30 msgr2=0x7f9bd019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0103b30 0x7f9bd019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 -- 172.21.15.37:0/2306071860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd019fdd0 con 0x7f9bd0101f80 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd67a2640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0103b30 0x7f9bd019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd5fa1640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 0x7f9bd01981b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009fd0 tx=0x7f9bc000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0011470 con 0x7f9bd0101f80 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc0011ac0 con 0x7f9bd0101f80 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd01a00c0 con 0x7f9bd0101f80 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc001aab0 con 0x7f9bd0101f80 2024-04-23T18:20:19.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.136+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01a0610 con 0x7f9bd0101f80 2024-04-23T18:20:19.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.138+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd01092f0 con 0x7f9bd0101f80 2024-04-23T18:20:19.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.139+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9bc001ac50 con 0x7f9bd0101f80 2024-04-23T18:20:19.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.139+0000 7f9bceffd640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9bb80a3bb0 0x7f9bb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.140+0000 7f9bd57a0640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9bb80a3bb0 0x7f9bb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.140+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f9bc00d0cb0 con 0x7f9bd0101f80 2024-04-23T18:20:19.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.140+0000 7f9bd57a0640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9bb80a3bb0 0x7f9bb80a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40096c0 tx=0x7f9bc4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:19.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.143+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9bc0016030 con 0x7f9bd0101f80 2024-04-23T18:20:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.327+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9bd002d850 con 0x7f9bd0101f80 2024-04-23T18:20:19.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.328+0000 7f9bceffd640 1 -- 172.21.15.37:0/2306071860 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f9bc009d620 con 0x7f9bd0101f80 2024-04-23T18:20:19.322 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9bb80a3bb0 msgr2=0x7f9bb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9bb80a3bb0 0x7f9bb80a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40096c0 tx=0x7f9bc4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 msgr2=0x7f9bd01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 0x7f9bd01981b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009fd0 tx=0x7f9bc000c270 comp rx=0 tx=0).stop 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 shutdown_connections 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f9bb80a3bb0 0x7f9bb80a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bd0103b30 0x7f9bd019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd0103170 0x7f9bd01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 --2- 172.21.15.37:0/2306071860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd0101f80 0x7f9bd01981b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 >> 172.21.15.37:0/2306071860 conn(0x7f9bd00fd510 msgr2=0x7f9bd01064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 shutdown_connections 2024-04-23T18:20:19.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.330+0000 7f9bd8205640 1 -- 172.21.15.37:0/2306071860 wait complete. 2024-04-23T18:20:19.334 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:19 smithi037 ceph-mon[29404]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:20:19.334 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:19 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3611390694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:19.334 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:19 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2881992878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:19.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:20:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/4189975628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24105360 msgr2=0x7f4e24107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/4189975628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24105360 0x7f4e24107750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009900 tx=0x7f4e1802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/4189975628 shutdown_connections 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/4189975628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e24107c90 0x7f4e2410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/4189975628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24105360 0x7f4e24107750 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/4189975628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e24102a30 0x7f4e24104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/4189975628 >> 172.21.15.37:0/4189975628 conn(0x7f4e240fc440 msgr2=0x7f4e240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/4189975628 shutdown_connections 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.446+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/4189975628 wait complete. 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 Processor -- start 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 -- start start 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e24102a30 0x7f4e241981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e24105360 0x7f4e241986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 0x7f4e2419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2410daf0 con 0x7f4e24107c90 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2410d970 con 0x7f4e24105360 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e2a6ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2410dc70 con 0x7f4e24102a30 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e28c8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 0x7f4e2419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e28c8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 0x7f4e2419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47598/0 (socket says 172.21.15.37:47598) 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e28c8c640 1 -- 172.21.15.37:0/204095625 learned_addr learned my addr 172.21.15.37:0/204095625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e23fff640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e24102a30 0x7f4e241981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.447+0000 7f4e237fe640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e24105360 0x7f4e241986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e28c8c640 1 -- 172.21.15.37:0/204095625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e24102a30 msgr2=0x7f4e241981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e28c8c640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e24102a30 0x7f4e241981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e28c8c640 1 -- 172.21.15.37:0/204095625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e24105360 msgr2=0x7f4e241986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e28c8c640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e24105360 0x7f4e241986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e28c8c640 1 -- 172.21.15.37:0/204095625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2419fdd0 con 0x7f4e24107c90 2024-04-23T18:20:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e28c8c640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 0x7f4e2419f760 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00cce0 tx=0x7f4e1c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00f450 con 0x7f4e24107c90 2024-04-23T18:20:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e241a00c0 con 0x7f4e24107c90 2024-04-23T18:20:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c00faa0 con 0x7f4e24107c90 2024-04-23T18:20:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c018b70 con 0x7f4e24107c90 2024-04-23T18:20:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.448+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241a0590 con 0x7f4e24107c90 2024-04-23T18:20:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.449+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e240fdf30 con 0x7f4e24107c90 2024-04-23T18:20:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.450+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e1c018d10 con 0x7f4e24107c90 2024-04-23T18:20:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.451+0000 7f4e217fa640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4e0c0a3bb0 0x7f4e0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.451+0000 7f4e23fff640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4e0c0a3bb0 0x7f4e0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.451+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f4e1c0d0840 con 0x7f4e24107c90 2024-04-23T18:20:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.451+0000 7f4e23fff640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4e0c0a3bb0 0x7f4e0c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009bd0 tx=0x7f4e14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.455+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e1c014030 con 0x7f4e24107c90 2024-04-23T18:20:19.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:19 smithi172 ceph-mon[32456]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:20:19.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:19 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3611390694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:19.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:19 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2881992878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:19.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:19 smithi173 ceph-mon[32906]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:20:19.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:19 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3611390694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:19.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:19 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2881992878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:19.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.643+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e24062b50 con 0x7f4e24107c90 2024-04-23T18:20:19.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.643+0000 7f4e217fa640 1 -- 172.21.15.37:0/204095625 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f4e1c09d1b0 con 0x7f4e24107c90 2024-04-23T18:20:19.637 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:20:19.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4e0c0a3bb0 msgr2=0x7f4e0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4e0c0a3bb0 0x7f4e0c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009bd0 tx=0x7f4e14005f90 comp rx=0 tx=0).stop 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 msgr2=0x7f4e2419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 0x7f4e2419f760 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00cce0 tx=0x7f4e1c00a620 comp rx=0 tx=0).stop 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 shutdown_connections 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4e0c0a3bb0 0x7f4e0c0a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e24107c90 0x7f4e2419f760 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e24105360 0x7f4e241986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 --2- 172.21.15.37:0/204095625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e24102a30 0x7f4e241981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 >> 172.21.15.37:0/204095625 conn(0x7f4e240fc440 msgr2=0x7f4e24103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 shutdown_connections 2024-04-23T18:20:19.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.645+0000 7f4e2a6ef640 1 -- 172.21.15.37:0/204095625 wait complete. 2024-04-23T18:20:19.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 -- 172.21.15.37:0/249287245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 msgr2=0x7fede0107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 --2- 172.21.15.37:0/249287245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0107750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009ca0 tx=0x7fedd402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 -- 172.21.15.37:0/249287245 shutdown_connections 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 --2- 172.21.15.37:0/249287245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0107c90 0x7fede010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 --2- 172.21.15.37:0/249287245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0107750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 --2- 172.21.15.37:0/249287245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0102a30 0x7fede0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 -- 172.21.15.37:0/249287245 >> 172.21.15.37:0/249287245 conn(0x7fede00fc440 msgr2=0x7fede00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 -- 172.21.15.37:0/249287245 shutdown_connections 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 -- 172.21.15.37:0/249287245 wait complete. 2024-04-23T18:20:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.758+0000 7fede5a7b640 1 Processor -- start 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 -- start start 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0102a30 0x7fede0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0107c90 0x7fede019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fede010daf0 con 0x7fede0105360 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fede010d970 con 0x7fede0107c90 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7fede5a7b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fede010dc70 con 0x7fede0102a30 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47620/0 (socket says 172.21.15.37:47620) 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 -- 172.21.15.37:0/1664742479 learned_addr learned my addr 172.21.15.37:0/1664742479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddf7fe640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0102a30 0x7fede0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddffff640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0107c90 0x7fede019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 -- 172.21.15.37:0/1664742479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0102a30 msgr2=0x7fede0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0102a30 0x7fede0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 -- 172.21.15.37:0/1664742479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0107c90 msgr2=0x7fede019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0107c90 0x7fede019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 -- 172.21.15.37:0/1664742479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede019fe60 con 0x7fede0105360 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddf7fe640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0102a30 0x7fede0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddffff640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0107c90 0x7fede019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.759+0000 7feddeffd640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0198780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009430 tx=0x7fedd4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.760+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4003cb0 con 0x7fede0105360 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.760+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01a00f0 con 0x7fede0105360 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.760+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd4003e50 con 0x7fede0105360 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.760+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd402fc60 con 0x7fede0105360 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.760+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede01a0640 con 0x7fede0105360 2024-04-23T18:20:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.761+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede00fdf30 con 0x7fede0105360 2024-04-23T18:20:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.762+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fedd4004040 con 0x7fede0105360 2024-04-23T18:20:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.762+0000 7feddcff9640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fedc80a3bb0 0x7fedc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.762+0000 7feddf7fe640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fedc80a3bb0 0x7fedc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.762+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fedd40f1a90 con 0x7fede0105360 2024-04-23T18:20:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.763+0000 7feddf7fe640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fedc80a3bb0 0x7fedc80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fedd00058c0 tx=0x7fedd0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.767+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedd40be460 con 0x7fede0105360 2024-04-23T18:20:19.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.993+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fede0062b50 con 0x7fede0105360 2024-04-23T18:20:19.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.994+0000 7feddcff9640 1 -- 172.21.15.37:0/1664742479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7fedd40bdd60 con 0x7fede0105360 2024-04-23T18:20:19.988 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:19.988 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:18:40.084107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:20:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.995+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fedc80a3bb0 msgr2=0x7fedc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.995+0000 7fede5a7b640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fedc80a3bb0 0x7fedc80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fedd00058c0 tx=0x7fedd0005850 comp rx=0 tx=0).stop 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.995+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 msgr2=0x7fede0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.995+0000 7fede5a7b640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0198780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009430 tx=0x7fedd4031840 comp rx=0 tx=0).stop 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.995+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 shutdown_connections 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fedc80a3bb0 0x7fedc80a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede0107c90 0x7fede019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fede0105360 0x7fede0198780 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 --2- 172.21.15.37:0/1664742479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fede0102a30 0x7fede0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 >> 172.21.15.37:0/1664742479 conn(0x7fede00fc440 msgr2=0x7fede0103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 shutdown_connections 2024-04-23T18:20:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:19.996+0000 7fede5a7b640 1 -- 172.21.15.37:0/1664742479 wait complete. 2024-04-23T18:20:20.001 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:20.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 -- 172.21.15.37:0/2753713851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b4103b70 msgr2=0x7f01b410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 --2- 172.21.15.37:0/2753713851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b4103b70 0x7f01b410a400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009ca0 tx=0x7f01ac02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 -- 172.21.15.37:0/2753713851 shutdown_connections 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 --2- 172.21.15.37:0/2753713851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b4103b70 0x7f01b410a400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 --2- 172.21.15.37:0/2753713851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b41031b0 0x7f01b4103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 --2- 172.21.15.37:0/2753713851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4101fc0 0x7f01b41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 -- 172.21.15.37:0/2753713851 >> 172.21.15.37:0/2753713851 conn(0x7f01b40fd550 msgr2=0x7f01b40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 -- 172.21.15.37:0/2753713851 shutdown_connections 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 -- 172.21.15.37:0/2753713851 wait complete. 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.110+0000 7f01bb543640 1 Processor -- start 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 -- start start 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b4101fc0 0x7f01b4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 0x7f01b4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4103b70 0x7f01b4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01b410dac0 con 0x7f01b41031b0 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01b410d940 con 0x7f01b4101fc0 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01bb543640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f01b410dc40 con 0x7f01b4103b70 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 0x7f01b4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 0x7f01b4074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47638/0 (socket says 172.21.15.37:47638) 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 -- 172.21.15.37:0/2328191803 learned_addr learned my addr 172.21.15.37:0/2328191803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b9ae0640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4103b70 0x7f01b4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b92df640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b4101fc0 0x7f01b4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 -- 172.21.15.37:0/2328191803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4103b70 msgr2=0x7f01b4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4103b70 0x7f01b4074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 -- 172.21.15.37:0/2328191803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b4101fc0 msgr2=0x7f01b4075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b4101fc0 0x7f01b4075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 -- 172.21.15.37:0/2328191803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b4075110 con 0x7f01b41031b0 2024-04-23T18:20:20.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01b8ade640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 0x7f01b4074340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c890 tx=0x7f01b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.111+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0004220 con 0x7f01b41031b0 2024-04-23T18:20:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.112+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b41a22b0 con 0x7f01b41031b0 2024-04-23T18:20:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.112+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b00043c0 con 0x7f01b41031b0 2024-04-23T18:20:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.112+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0004a00 con 0x7f01b41031b0 2024-04-23T18:20:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.112+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b41a2850 con 0x7f01b41031b0 2024-04-23T18:20:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.113+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b4109380 con 0x7f01b41031b0 2024-04-23T18:20:20.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.114+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f01b00079b0 con 0x7f01b41031b0 2024-04-23T18:20:20.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.114+0000 7f01aa7fc640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f01a00a3bb0 0x7f01a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.114+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f01b00d01b0 con 0x7f01b41031b0 2024-04-23T18:20:20.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.114+0000 7f01b92df640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f01a00a3bb0 0x7f01a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.115+0000 7f01b92df640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f01a00a3bb0 0x7f01a00a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.119+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01b0014030 con 0x7f01b41031b0 2024-04-23T18:20:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.306+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01b4062b50 con 0x7f01b41031b0 2024-04-23T18:20:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.308+0000 7f01aa7fc640 1 -- 172.21.15.37:0/2328191803 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22684 (secure 0 0 0) 0x7f01b009cb80 con 0x7f01b41031b0 2024-04-23T18:20:20.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:20.303 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:19:41.678555+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f01a00a3bb0 msgr2=0x7f01a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f01a00a3bb0 0x7f01a00a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4005850 comp rx=0 tx=0).stop 2024-04-23T18:20:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 msgr2=0x7f01b4074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 0x7f01b4074340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c890 tx=0x7f01b000cd50 comp rx=0 tx=0).stop 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 shutdown_connections 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f01a00a3bb0 0x7f01a00a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b4103b70 0x7f01b4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b41031b0 0x7f01b4074340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 --2- 172.21.15.37:0/2328191803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b4101fc0 0x7f01b4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 >> 172.21.15.37:0/2328191803 conn(0x7f01b40fd550 msgr2=0x7f01b4106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 shutdown_connections 2024-04-23T18:20:20.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.310+0000 7f01bb543640 1 -- 172.21.15.37:0/2328191803 wait complete. 2024-04-23T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:20 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2306071860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:20 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/204095625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:20 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1664742479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:20.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:20:20.316 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 -- 172.21.15.37:0/1160921622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21341029a0 msgr2=0x7f21341092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 --2- 172.21.15.37:0/1160921622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21341029a0 0x7f21341092c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f212c009ca0 tx=0x7f212c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 -- 172.21.15.37:0/1160921622 shutdown_connections 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 --2- 172.21.15.37:0/1160921622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21341029a0 0x7f21341092c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 --2- 172.21.15.37:0/1160921622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 0x7f2134102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 --2- 172.21.15.37:0/1160921622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2134100e90 0x7f21341012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 -- 172.21.15.37:0/1160921622 >> 172.21.15.37:0/1160921622 conn(0x7f21340fc400 msgr2=0x7f21340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 -- 172.21.15.37:0/1160921622 shutdown_connections 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.427+0000 7f213b8cf640 1 -- 172.21.15.37:0/1160921622 wait complete. 2024-04-23T18:20:20.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 Processor -- start 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 -- start start 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 0x7f2134069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 0x7f213419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21341029a0 0x7f213419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f213410ca20 con 0x7f2134100e90 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f213410c8a0 con 0x7f2134101fe0 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213b8cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f213410cba0 con 0x7f21341029a0 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 0x7f2134069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 0x7f2134069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47648/0 (socket says 172.21.15.37:47648) 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 -- 172.21.15.37:0/2876468674 learned_addr learned my addr 172.21.15.37:0/2876468674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f2138e6a640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 0x7f213419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 -- 172.21.15.37:0/2876468674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21341029a0 msgr2=0x7f213419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21341029a0 0x7f213419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 -- 172.21.15.37:0/2876468674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 msgr2=0x7f213419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 0x7f213419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.428+0000 7f213966b640 1 -- 172.21.15.37:0/2876468674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213419fde0 con 0x7f2134100e90 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f2138e6a640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 0x7f213419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f213966b640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 0x7f2134069c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f213000d950 tx=0x7f213000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130017030 con 0x7f2134100e90 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2130004020 con 0x7f2134100e90 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21341a0070 con 0x7f2134100e90 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21300047a0 con 0x7f2134100e90 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.429+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21341a0610 con 0x7f2134100e90 2024-04-23T18:20:20.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.430+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21341081a0 con 0x7f2134100e90 2024-04-23T18:20:20.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.431+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2130009a30 con 0x7f2134100e90 2024-04-23T18:20:20.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.432+0000 7f21267fc640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f21200a3c30 0x7f21200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.432+0000 7f2138e6a640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f21200a3c30 0x7f21200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.432+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f21300d05e0 con 0x7f2134100e90 2024-04-23T18:20:20.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.432+0000 7f2138e6a640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f21200a3c30 0x7f21200a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f21340699c0 tx=0x7f2128005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:20.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.436+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f213009cfd0 con 0x7f2134100e90 2024-04-23T18:20:20.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:20 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2306071860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:20.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:20 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/204095625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:20.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:20 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1664742479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:20.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:20 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2306071860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:20.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:20 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/204095625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:20.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:20 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1664742479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:20.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.622+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f213402d870 con 0x7f2134100e90 2024-04-23T18:20:20.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.622+0000 7f21267fc640 1 -- 172.21.15.37:0/2876468674 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f213009c8d0 con 0x7f2134100e90 2024-04-23T18:20:20.617 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:20:20.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f21200a3c30 msgr2=0x7f21200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f21200a3c30 0x7f21200a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f21340699c0 tx=0x7f2128005e50 comp rx=0 tx=0).stop 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 msgr2=0x7f2134069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 0x7f2134069c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f213000d950 tx=0x7f213000de10 comp rx=0 tx=0).stop 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 shutdown_connections 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f21200a3c30 0x7f21200a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21341029a0 0x7f213419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2134101fe0 0x7f213419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 --2- 172.21.15.37:0/2876468674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2134100e90 0x7f2134069c50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 >> 172.21.15.37:0/2876468674 conn(0x7f21340fc400 msgr2=0x7f2134105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 shutdown_connections 2024-04-23T18:20:20.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.624+0000 7f213b8cf640 1 -- 172.21.15.37:0/2876468674 wait complete. 2024-04-23T18:20:20.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.740+0000 7fba08059640 1 -- 172.21.15.37:0/3236373335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00101f80 msgr2=0x7fba001023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.740+0000 7fba08059640 1 --2- 172.21.15.37:0/3236373335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00101f80 0x7fba001023a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009900 tx=0x7fb9fc02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- 172.21.15.37:0/3236373335 shutdown_connections 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 --2- 172.21.15.37:0/3236373335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba00103b30 0x7fba0010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 --2- 172.21.15.37:0/3236373335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba00103170 0x7fba001035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 --2- 172.21.15.37:0/3236373335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00101f80 0x7fba001023a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- 172.21.15.37:0/3236373335 >> 172.21.15.37:0/3236373335 conn(0x7fba000fd510 msgr2=0x7fba000ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- 172.21.15.37:0/3236373335 shutdown_connections 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- 172.21.15.37:0/3236373335 wait complete. 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 Processor -- start 2024-04-23T18:20:20.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- start start 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba00101f80 0x7fba00075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba00103170 0x7fba00074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 0x7fba00074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba0010dac0 con 0x7fba00103b30 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba0010d940 con 0x7fba00103170 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.741+0000 7fba08059640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba0010dc40 con 0x7fba00101f80 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 0x7fba00074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 0x7fba00074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47658/0 (socket says 172.21.15.37:47658) 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 -- 172.21.15.37:0/414439027 learned_addr learned my addr 172.21.15.37:0/414439027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba05df5640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba00101f80 0x7fba00075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba055f4640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba00103170 0x7fba00074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 -- 172.21.15.37:0/414439027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba00101f80 msgr2=0x7fba00075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba00101f80 0x7fba00075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 -- 172.21.15.37:0/414439027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba00103170 msgr2=0x7fba00074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba00103170 0x7fba00074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 -- 172.21.15.37:0/414439027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba00075110 con 0x7fba00103b30 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba065f6640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 0x7fba00074880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800cce0 tx=0x7fb9f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800f450 con 0x7fba00103b30 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001a2210 con 0x7fba00103b30 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f800faa0 con 0x7fba00103b30 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8018b70 con 0x7fba00103b30 2024-04-23T18:20:20.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.742+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a2760 con 0x7fba00103b30 2024-04-23T18:20:20.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.743+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba001092a0 con 0x7fba00103b30 2024-04-23T18:20:20.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.744+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9f8018d10 con 0x7fba00103b30 2024-04-23T18:20:20.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.745+0000 7fb9f6ffd640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9ec0a3bb0 0x7fb9ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:20.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.745+0000 7fba05df5640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9ec0a3bb0 0x7fb9ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:20.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.745+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fb9f80d0980 con 0x7fba00103b30 2024-04-23T18:20:20.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.745+0000 7fba05df5640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9ec0a3bb0 0x7fb9ec0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:20.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.749+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9f8014030 con 0x7fba00103b30 2024-04-23T18:20:20.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.937+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fba00062b50 con 0x7fba00103b30 2024-04-23T18:20:20.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.937+0000 7fb9f6ffd640 1 -- 172.21.15.37:0/414439027 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fb9f809d350 con 0x7fba00103b30 2024-04-23T18:20:20.931 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9ec0a3bb0 msgr2=0x7fb9ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9ec0a3bb0 0x7fb9ec0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc0314a0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 msgr2=0x7fba00074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 0x7fba00074880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800cce0 tx=0x7fb9f800a620 comp rx=0 tx=0).stop 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 shutdown_connections 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fb9ec0a3bb0 0x7fb9ec0a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba00103b30 0x7fba00074880 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba00103170 0x7fba00074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 --2- 172.21.15.37:0/414439027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba00101f80 0x7fba00075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:20.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 >> 172.21.15.37:0/414439027 conn(0x7fba000fd510 msgr2=0x7fba00106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:20.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 shutdown_connections 2024-04-23T18:20:20.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:20.939+0000 7fba08059640 1 -- 172.21.15.37:0/414439027 wait complete. 2024-04-23T18:20:20.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:20:21.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 -- 172.21.15.37:0/3960631731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 msgr2=0x7fbdbc10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 --2- 172.21.15.37:0/3960631731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc10a120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009ca0 tx=0x7fbdb402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 -- 172.21.15.37:0/3960631731 shutdown_connections 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 --2- 172.21.15.37:0/3960631731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc10a120 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 --2- 172.21.15.37:0/3960631731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 0x7fbdbc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 --2- 172.21.15.37:0/3960631731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdbc102ad0 0x7fbdbc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 -- 172.21.15.37:0/3960631731 >> 172.21.15.37:0/3960631731 conn(0x7fbdbc0fc440 msgr2=0x7fbdbc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 -- 172.21.15.37:0/3960631731 shutdown_connections 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.055+0000 7fbdc2376640 1 -- 172.21.15.37:0/3960631731 wait complete. 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 Processor -- start 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 -- start start 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdbc102ad0 0x7fbdbc198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 0x7fbdbc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc10db10 con 0x7fbdbc107d30 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc10d990 con 0x7fbdbc102ad0 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc2376640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc10dc90 con 0x7fbdbc105400 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc0913640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc0913640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47678/0 (socket says 172.21.15.37:47678) 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdc0913640 1 -- 172.21.15.37:0/2428510466 learned_addr learned my addr 172.21.15.37:0/2428510466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdbbfff640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdbc102ad0 0x7fbdbc198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.056+0000 7fbdbb7fe640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 0x7fbdbc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc0913640 1 -- 172.21.15.37:0/2428510466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 msgr2=0x7fbdbc198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc0913640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 0x7fbdbc198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc0913640 1 -- 172.21.15.37:0/2428510466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdbc102ad0 msgr2=0x7fbdbc198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc0913640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdbc102ad0 0x7fbdbc198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc0913640 1 -- 172.21.15.37:0/2428510466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc19fd40 con 0x7fbdbc107d30 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdbb7fe640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 0x7fbdbc198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc0913640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc19f6d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40097b0 tx=0x7fbdb4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4004240 con 0x7fbdbc107d30 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdbc19ffd0 con 0x7fbdbc107d30 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb40317c0 con 0x7fbdbc107d30 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb402fd80 con 0x7fbdbc107d30 2024-04-23T18:20:21.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.057+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdbc1a0490 con 0x7fbdbc107d30 2024-04-23T18:20:21.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.058+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdbc0fdf30 con 0x7fbdbc107d30 2024-04-23T18:20:21.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.059+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbdb40043e0 con 0x7fbdbc107d30 2024-04-23T18:20:21.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.060+0000 7fbdb97fa640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbda80a3bb0 0x7fbda80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.060+0000 7fbdbbfff640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbda80a3bb0 0x7fbda80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.060+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fbdb40f1f30 con 0x7fbdbc107d30 2024-04-23T18:20:21.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.060+0000 7fbdbbfff640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbda80a3bb0 0x7fbda80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbda400bbe0 tx=0x7fbda400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:21.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.063+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdb407fff0 con 0x7fbdbc107d30 2024-04-23T18:20:21.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.249+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbdbc02d810 con 0x7fbdbc107d30 2024-04-23T18:20:21.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.249+0000 7fbdb97fa640 1 -- 172.21.15.37:0/2428510466 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fbdb40bea20 con 0x7fbdbc107d30 2024-04-23T18:20:21.243 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbda80a3bb0 msgr2=0x7fbda80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbda80a3bb0 0x7fbda80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbda400bbe0 tx=0x7fbda400b390 comp rx=0 tx=0).stop 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 msgr2=0x7fbdbc19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc19f6d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40097b0 tx=0x7fbdb4003c20 comp rx=0 tx=0).stop 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 shutdown_connections 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fbda80a3bb0 0x7fbda80a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdbc107d30 0x7fbdbc19f6d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdbc105400 0x7fbdbc198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 --2- 172.21.15.37:0/2428510466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdbc102ad0 0x7fbdbc198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 >> 172.21.15.37:0/2428510466 conn(0x7fbdbc0fc440 msgr2=0x7fbdbc103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:21.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 shutdown_connections 2024-04-23T18:20:21.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.251+0000 7fbdc2376640 1 -- 172.21.15.37:0/2428510466 wait complete. 2024-04-23T18:20:21.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:21 smithi037 ceph-mon[29404]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:20:21.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:21 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2328191803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:21.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:21 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2876468674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:21.256 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:21 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/414439027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:21.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:20:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 -- 172.21.15.37:0/1743712078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc107c90 msgr2=0x7f4ccc10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/1743712078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc107c90 0x7f4ccc10a0e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009ca0 tx=0x7f4cc402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 -- 172.21.15.37:0/1743712078 shutdown_connections 2024-04-23T18:20:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/1743712078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc107c90 0x7f4ccc10a0e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/1743712078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ccc105360 0x7f4ccc107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/1743712078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ccc102a30 0x7f4ccc104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 -- 172.21.15.37:0/1743712078 >> 172.21.15.37:0/1743712078 conn(0x7f4ccc0fc440 msgr2=0x7f4ccc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 -- 172.21.15.37:0/1743712078 shutdown_connections 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.364+0000 7f4cd12e1640 1 -- 172.21.15.37:0/1743712078 wait complete. 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 Processor -- start 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 -- start start 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 0x7f4ccc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ccc105360 0x7f4ccc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ccc107c90 0x7f4ccc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc10db50 con 0x7f4ccc102a30 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc10d9d0 con 0x7f4ccc105360 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cd12e1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc10dcd0 con 0x7f4ccc107c90 2024-04-23T18:20:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 0x7f4ccc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 0x7f4ccc1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47700/0 (socket says 172.21.15.37:47700) 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 -- 172.21.15.37:0/2578918195 learned_addr learned my addr 172.21.15.37:0/2578918195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccb7fe640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ccc107c90 0x7f4ccc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4cca7fc640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ccc105360 0x7f4ccc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 -- 172.21.15.37:0/2578918195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ccc107c90 msgr2=0x7f4ccc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ccc107c90 0x7f4ccc19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 -- 172.21.15.37:0/2578918195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ccc105360 msgr2=0x7f4ccc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ccc105360 0x7f4ccc1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.365+0000 7f4ccaffd640 1 -- 172.21.15.37:0/2578918195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc19fdc0 con 0x7f4ccc102a30 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.366+0000 7f4ccaffd640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 0x7f4ccc1981a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b4a0 tx=0x7f4cc000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.366+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0014070 con 0x7f4ccc102a30 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.366+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc1a00b0 con 0x7f4ccc102a30 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.366+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc0010ac0 con 0x7f4ccc102a30 2024-04-23T18:20:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.366+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0017c30 con 0x7f4ccc102a30 2024-04-23T18:20:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.366+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc1a0630 con 0x7f4ccc102a30 2024-04-23T18:20:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.367+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ccc0fdf30 con 0x7f4ccc102a30 2024-04-23T18:20:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.368+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4cc0017070 con 0x7f4ccc102a30 2024-04-23T18:20:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.369+0000 7f4cb3fff640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4cb40a3bb0 0x7f4cb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.369+0000 7f4cca7fc640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4cb40a3bb0 0x7f4cb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.369+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f4cc00cf8c0 con 0x7f4ccc102a30 2024-04-23T18:20:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.369+0000 7f4cca7fc640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4cb40a3bb0 0x7f4cb40a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc1996b0 tx=0x7f4cbc005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.373+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cc009c230 con 0x7f4ccc102a30 2024-04-23T18:20:21.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:21 smithi173 ceph-mon[32906]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:20:21.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:21 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2328191803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:21.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:21 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2876468674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:21.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:21 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/414439027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:21.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:21 smithi172 ceph-mon[32456]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:20:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:21 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2328191803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:21 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2876468674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:21.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:21 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/414439027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.601+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ccc062b50 con 0x7f4ccc102a30 2024-04-23T18:20:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.602+0000 7f4cb3fff640 1 -- 172.21.15.37:0/2578918195 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f4cc009bb30 con 0x7f4ccc102a30 2024-04-23T18:20:21.596 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:21.596 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:18:40.084107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:20:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-23T18:20:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4cb40a3bb0 msgr2=0x7f4cb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4cb40a3bb0 0x7f4cb40a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc1996b0 tx=0x7f4cbc005cc0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 msgr2=0x7f4ccc1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 0x7f4ccc1981a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b4a0 tx=0x7f4cc000b960 comp rx=0 tx=0).stop 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 shutdown_connections 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f4cb40a3bb0 0x7f4cb40a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ccc107c90 0x7f4ccc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ccc105360 0x7f4ccc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 --2- 172.21.15.37:0/2578918195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc102a30 0x7f4ccc1981a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 >> 172.21.15.37:0/2578918195 conn(0x7f4ccc0fc440 msgr2=0x7f4ccc1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 shutdown_connections 2024-04-23T18:20:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.604+0000 7f4cd12e1640 1 -- 172.21.15.37:0/2578918195 wait complete. 2024-04-23T18:20:21.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- 172.21.15.37:0/3912804772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 msgr2=0x7f96bc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/3912804772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1024c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009900 tx=0x7f96a802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- 172.21.15.37:0/3912804772 shutdown_connections 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/3912804772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96bc102a00 0x7f96bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/3912804772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1024c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/3912804772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc100e60 0x7f96bc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- 172.21.15.37:0/3912804772 >> 172.21.15.37:0/3912804772 conn(0x7f96bc0fc440 msgr2=0x7f96bc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- 172.21.15.37:0/3912804772 shutdown_connections 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- 172.21.15.37:0/3912804772 wait complete. 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 Processor -- start 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- start start 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96bc100e60 0x7f96bc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc102a00 0x7f96bc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc10ca80 con 0x7f96bc102040 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc10c900 con 0x7f96bc100e60 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96c2e0b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc10cc00 con 0x7f96bc102a00 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.718+0000 7f96bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47724/0 (socket says 172.21.15.37:47724) 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 -- 172.21.15.37:0/1055783664 learned_addr learned my addr 172.21.15.37:0/1055783664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96c13a8640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc102a00 0x7f96bc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96c0ba7640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96bc100e60 0x7f96bc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 -- 172.21.15.37:0/1055783664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc102a00 msgr2=0x7f96bc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc102a00 0x7f96bc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 -- 172.21.15.37:0/1055783664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96bc100e60 msgr2=0x7f96bc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96bc100e60 0x7f96bc1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 -- 172.21.15.37:0/1055783664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc19fe70 con 0x7f96bc102040 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96bbfff640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1987e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005940 tx=0x7f96a8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a803f030 con 0x7f96bc102040 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc1a0100 con 0x7f96bc102040 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a8003cb0 con 0x7f96bc102040 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8035070 con 0x7f96bc102040 2024-04-23T18:20:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.719+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc1a0670 con 0x7f96bc102040 2024-04-23T18:20:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.720+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96bc1a08d0 con 0x7f96bc102040 2024-04-23T18:20:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.722+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f96a8042030 con 0x7f96bc102040 2024-04-23T18:20:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.722+0000 7f96b9ffb640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f96ac0a3bb0 0x7f96ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.722+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f96a80f0fe0 con 0x7f96bc102040 2024-04-23T18:20:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.722+0000 7f96c0ba7640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f96ac0a3bb0 0x7f96ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.723+0000 7f96c0ba7640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f96ac0a3bb0 0x7f96ac0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96b0005fd0 tx=0x7f96b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.726+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96a80bd9b0 con 0x7f96bc102040 2024-04-23T18:20:21.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.914+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96bc1a0aa0 con 0x7f96bc102040 2024-04-23T18:20:21.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.915+0000 7f96b9ffb640 1 -- 172.21.15.37:0/1055783664 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22684 (secure 0 0 0) 0x7f96a80b7e20 con 0x7f96bc102040 2024-04-23T18:20:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:21.910 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:19:41.678555+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f96ac0a3bb0 msgr2=0x7f96ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f96ac0a3bb0 0x7f96ac0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96b0005fd0 tx=0x7f96b0005850 comp rx=0 tx=0).stop 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 msgr2=0x7f96bc1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1987e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005940 tx=0x7f96a8004430 comp rx=0 tx=0).stop 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 shutdown_connections 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f96ac0a3bb0 0x7f96ac0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc102a00 0x7f96bc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96bc102040 0x7f96bc1987e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 --2- 172.21.15.37:0/1055783664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96bc100e60 0x7f96bc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 >> 172.21.15.37:0/1055783664 conn(0x7f96bc0fc440 msgr2=0x7f96bc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 shutdown_connections 2024-04-23T18:20:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:21.917+0000 7f96c2e0b640 1 -- 172.21.15.37:0/1055783664 wait complete. 2024-04-23T18:20:21.924 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:18:34.408115+0000', 'modified': '2024-04-23T18:18:40.084107+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24484, 'mds_2': 24479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2640036963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2640036963}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2640036963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2419231317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2419231317}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2419231317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6837/3035435067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3035435067}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3035435067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.172:6837/364708999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 364708999}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 364708999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.173:6839/496716250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 496716250}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 496716250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6839/2128854391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 2128854391}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 2128854391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24479, 24484, 24509]} 2024-04-23T18:20:21.924 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-23T18:20:21.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:20:22.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 -- 172.21.15.37:0/3484601311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102080 msgr2=0x7f38b8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 --2- 172.21.15.37:0/3484601311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102080 0x7f38b8102500 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009900 tx=0x7f38a802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:22.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 -- 172.21.15.37:0/3484601311 shutdown_connections 2024-04-23T18:20:22.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 --2- 172.21.15.37:0/3484601311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38b8102a40 0x7f38b8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 --2- 172.21.15.37:0/3484601311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102080 0x7f38b8102500 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 --2- 172.21.15.37:0/3484601311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b8100f30 0x7f38b8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 -- 172.21.15.37:0/3484601311 >> 172.21.15.37:0/3484601311 conn(0x7f38b80fc4a0 msgr2=0x7f38b80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 -- 172.21.15.37:0/3484601311 shutdown_connections 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 -- 172.21.15.37:0/3484601311 wait complete. 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 Processor -- start 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.033+0000 7f38bff44640 1 -- start start 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38b8100f30 0x7f38b8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b8102080 0x7f38b8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 0x7f38b819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810dbd0 con 0x7f38b8102a40 2024-04-23T18:20:22.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810da50 con 0x7f38b8102080 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810dd50 con 0x7f38b8100f30 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 0x7f38b819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 0x7f38b819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47738/0 (socket says 172.21.15.37:47738) 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 -- 172.21.15.37:0/3786737291 learned_addr learned my addr 172.21.15.37:0/3786737291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bd4df640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b8102080 0x7f38b8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bdce0640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38b8100f30 0x7f38b8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 -- 172.21.15.37:0/3786737291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38b8100f30 msgr2=0x7f38b8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38b8100f30 0x7f38b8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 -- 172.21.15.37:0/3786737291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b8102080 msgr2=0x7f38b8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b8102080 0x7f38b8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 -- 172.21.15.37:0/3786737291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b819fe30 con 0x7f38b8102a40 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38be4e1640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 0x7f38b819f7c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f38b000eb10 tx=0x7f38b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38aeffd640 1 -- 172.21.15.37:0/3786737291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0004b20 con 0x7f38b8102a40 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b81a0120 con 0x7f38b8102a40 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38aeffd640 1 -- 172.21.15.37:0/3786737291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b0004cc0 con 0x7f38b8102a40 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.034+0000 7f38aeffd640 1 -- 172.21.15.37:0/3786737291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0010690 con 0x7f38b8102a40 2024-04-23T18:20:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.035+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81a0690 con 0x7f38b8102a40 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38aeffd640 1 -- 172.21.15.37:0/3786737291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38b0005020 con 0x7f38b8102a40 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 Processor -- start 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 -- start start 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e55f90 0x5567a1e563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567a1db6c50 con 0x5567a1e57b20 2024-04-23T18:20:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5567a1d53830 con 0x5567a1e55f90 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38c2c63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5567a1db2c10 con 0x5567a1e57160 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38aeffd640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f38a40a3bb0 0x7f38a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e5e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47754/0 (socket says 172.21.15.37:47754) 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.037+0000 7f38bd4df640 1 -- 172.21.15.37:0/2961694910 learned_addr learned my addr 172.21.15.37:0/2961694910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38be4e1640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f38a40a3bb0 0x7f38a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e55f90 0x5567a1e563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bdce0640 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38aeffd640 1 -- 172.21.15.37:0/3786737291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f38b00d0950 con 0x7f38b8102a40 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 -- 172.21.15.37:0/2961694910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 msgr2=0x5567a1e575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 -- 172.21.15.37:0/2961694910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e55f90 msgr2=0x5567a1e563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e55f90 0x5567a1e563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 -- 172.21.15.37:0/2961694910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a1ac92d0 con 0x5567a1e57b20 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38be4e1640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f38a40a3bb0 0x7f38a40a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f38b00995f0 tx=0x7f38b006f760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e5e3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f38a8004750 tx=0x7f38a80046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327228be7fd9f99c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38a3fff640 1 -- 172.21.15.37:0/2961694910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8035040 con 0x5567a1e57b20 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38a3fff640 1 -- 172.21.15.37:0/2961694910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8031650 con 0x5567a1e57b20 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.038+0000 7f38a3fff640 1 -- 172.21.15.37:0/2961694910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8003a50 con 0x5567a1e57b20 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 -- 172.21.15.37:0/2961694910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 msgr2=0x5567a1e5e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e5e3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f38a8004750 tx=0x7f38a80046e0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 -- 172.21.15.37:0/2961694910 shutdown_connections 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e5e3b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2961694910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e55f90 0x5567a1e563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 -- 172.21.15.37:0/2961694910 >> 172.21.15.37:0/2961694910 conn(0x5567a1e4d070 msgr2=0x5567a1e4d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 -- 172.21.15.37:0/2961694910 shutdown_connections 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 -- 172.21.15.37:0/2961694910 wait complete. 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 -- start start 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.039+0000 7f38c2c63740 1 Processor -- start 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e60470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57b20 0x5567a1e679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e5ead0 con 0x5567a1e55f90 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5567a1d51470 con 0x5567a1e57160 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e5ec50 con 0x5567a1e57b20 2024-04-23T18:20:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38c2c63740 10 client.0 _ll_register_callbacks cb 0x5567a1d76f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e60470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e60470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47764/0 (socket says 172.21.15.37:47764) 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 -- 172.21.15.37:0/3488060584 learned_addr learned my addr 172.21.15.37:0/3488060584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bdce0640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57b20 0x5567a1e679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 -- 172.21.15.37:0/3488060584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57b20 msgr2=0x5567a1e679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57b20 0x5567a1e679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 -- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 msgr2=0x5567a1e609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 -- 172.21.15.37:0/3488060584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a1b936b0 con 0x5567a1e55f90 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bdce0640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38bd4df640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e60470 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38a802a930 tx=0x7f38a8042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3fa1072fcf3686 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.040+0000 7f38a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38a17fa640 1 -- 172.21.15.37:0/3488060584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8052060 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38a17fa640 1 -- 172.21.15.37:0/3488060584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f38a803e970 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a8031650 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38a17fa640 1 -- 172.21.15.37:0/3488060584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f38a8035040 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 10 client.15204 fetch_fsmap learned FSMap version 22 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5567a1e5ead0 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38a17fa640 1 -- 172.21.15.37:0/3488060584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8031650 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38a17fa640 1 -- 172.21.15.37:0/3488060584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f38a804bba0 con 0x5567a1e55f90 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 10 client.15204 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 20 client.15204 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1e6ce30 0x5567a1e6e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 4 client.15204 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.041+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567a1dc4670 con 0x5567a1e6ce30 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.042+0000 7f38bdce0640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1e6ce30 0x5567a1e6e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.042+0000 7f38bdce0640 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1e6ce30 0x5567a1e6e2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2aa8e425419190c0 server_cookie=5d92647475ba0dc1 in_seq=0 out_seq=0 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.042+0000 7f38a17fa640 10 client.15204 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38a17fa640 1 -- 172.21.15.37:0/3488060584 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567a1dc4670 con 0x5567a1e6ce30 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38a17fa640 10 client.15204 handle_command_reply: tid=0 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 client.15204 shutdown 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 msgr2=0x5567a1e60470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e60470 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38a802a930 tx=0x7f38a8042040 comp rx=0 tx=0).stop 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 shutdown_connections 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1e6ce30 0x5567a1e6e2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57b20 0x5567a1e679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 --2- 172.21.15.37:0/3488060584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e60470 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38a17fa640 0 client.15204 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 >> 172.21.15.37:0/3488060584 conn(0x5567a1e4d070 msgr2=0x5567a1e4c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 shutdown_connections 2024-04-23T18:20:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 1 -- 172.21.15.37:0/3488060584 wait complete. 2024-04-23T18:20:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.043+0000 7f38c2c63740 20 client.15204 trim_cache size 0 max 16384 2024-04-23T18:20:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.061+0000 7f38c2c63740 1 Processor -- start 2024-04-23T18:20:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.061+0000 7f38c2c63740 1 -- start start 2024-04-23T18:20:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e6bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e5a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57b20 0x5567a1e480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38c2c63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e6c5e0 con 0x5567a1e55f90 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38c2c63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e6ccb0 con 0x5567a1e57b20 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38c2c63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e6d380 con 0x5567a1e57160 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e6bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e6bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47768/0 (socket says 172.21.15.37:47768) 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 -- 172.21.15.37:0/2453070138 learned_addr learned my addr 172.21.15.37:0/2453070138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 -- 172.21.15.37:0/2453070138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 msgr2=0x5567a1e5a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bdce0640 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e5a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e5a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 -- 172.21.15.37:0/2453070138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57b20 msgr2=0x5567a1e480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57b20 0x5567a1e480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 -- 172.21.15.37:0/2453070138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a1e489d0 con 0x5567a1e55f90 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bdce0640 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e5a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38bd4df640 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e6bee0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005940 tx=0x7f38a80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22c43ca877eeb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38a37fe640 1 -- 172.21.15.37:0/2453070138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8052070 con 0x5567a1e55f90 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38a37fe640 1 -- 172.21.15.37:0/2453070138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8058700 con 0x5567a1e55f90 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.062+0000 7f38a37fe640 1 -- 172.21.15.37:0/2453070138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8058c60 con 0x5567a1e55f90 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 -- 172.21.15.37:0/2453070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 msgr2=0x5567a1e6bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e6bee0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38a8005940 tx=0x7f38a80095e0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 -- 172.21.15.37:0/2453070138 shutdown_connections 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57b20 0x5567a1e480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e57160 0x5567a1e5a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 --2- 172.21.15.37:0/2453070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e55f90 0x5567a1e6bee0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 -- 172.21.15.37:0/2453070138 >> 172.21.15.37:0/2453070138 conn(0x5567a1e4d070 msgr2=0x5567a1e4d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 -- 172.21.15.37:0/2453070138 shutdown_connections 2024-04-23T18:20:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 -- 172.21.15.37:0/2453070138 wait complete. 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 -- start start 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.063+0000 7f38c2c63740 1 Processor -- start 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e55f90 0x5567a1e5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e5cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e6e120 con 0x5567a1e57b20 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e6da50 con 0x5567a1e57160 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5567a1e6e7f0 con 0x5567a1e55f90 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 10 client.0 _ll_register_callbacks cb 0x7f38a8032c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e65a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47784/0 (socket says 172.21.15.37:47784) 2024-04-23T18:20:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 -- 172.21.15.37:0/1457810853 learned_addr learned my addr 172.21.15.37:0/1457810853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bdce0640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e5cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e55f90 0x5567a1e5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 -- 172.21.15.37:0/1457810853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e55f90 msgr2=0x5567a1e5c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e55f90 0x5567a1e5c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 -- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 msgr2=0x5567a1e5cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e5cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 -- 172.21.15.37:0/1457810853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a1e660c0 con 0x5567a1e57b20 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38bd4df640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e65a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38a80097a0 tx=0x7f38a80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ac3065d68dfc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38a17fa640 1 -- 172.21.15.37:0/1457810853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8052070 con 0x5567a1e57b20 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38a17fa640 1 -- 172.21.15.37:0/1457810853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f38a803e240 con 0x5567a1e57b20 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.064+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a8058700 con 0x5567a1e57b20 2024-04-23T18:20:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38a17fa640 1 -- 172.21.15.37:0/1457810853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f38a8004020 con 0x5567a1e57b20 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 10 client.15210 fetch_fsmap learned FSMap version 22 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5567a1ed2de0 con 0x5567a1e57b20 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38a17fa640 1 -- 172.21.15.37:0/1457810853 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8058700 con 0x5567a1e57b20 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38a17fa640 1 -- 172.21.15.37:0/1457810853 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f38a80035f0 con 0x5567a1e57b20 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 10 client.15210 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 20 client.15210 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1ed7400 0x5567a1ed3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 4 client.15210 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "status"}) -- 0x5567a1dbe4e0 con 0x5567a1ed7400 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.065+0000 7f38bdce0640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1ed7400 0x5567a1ed3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38a17fa640 10 client.15210 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38bdce0640 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1ed7400 0x5567a1ed3f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a8b8278d0186154 server_cookie=f400576b34d913ff in_seq=0 out_seq=0 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38a17fa640 1 -- 172.21.15.37:0/1457810853 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5567a1dbe4e0 con 0x5567a1ed7400 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38a17fa640 10 client.15210 handle_command_reply: tid=0 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 client.15210 shutdown 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 msgr2=0x5567a1e65a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e65a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38a80097a0 tx=0x7f38a80057b0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 shutdown_connections 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x5567a1ed7400 0x5567a1ed3f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567a1e57b20 0x5567a1e65a80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567a1e57160 0x5567a1e5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 --2- 172.21.15.37:0/1457810853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567a1e55f90 0x5567a1e5c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38a17fa640 0 client.15210 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 >> 172.21.15.37:0/1457810853 conn(0x5567a1e4d070 msgr2=0x5567a1e5def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 shutdown_connections 2024-04-23T18:20:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.066+0000 7f38c2c63740 1 -- 172.21.15.37:0/1457810853 wait complete. 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38c2c63740 20 client.15210 trim_cache size 0 max 16384 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24479, 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 102.993376779, 2024-04-23T18:20:22.068 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-04-23T18:20:22.069 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-23T18:20:22.069 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:20:22.069 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 124.521238909 2024-04-23T18:20:22.069 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f38a40a3bb0 msgr2=0x7f38a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f38a40a3bb0 0x7f38a40a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f38b00995f0 tx=0x7f38b006f760 comp rx=0 tx=0).stop 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 msgr2=0x7f38b819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 0x7f38b819f7c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f38b000eb10 tx=0x7f38b000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 shutdown_connections 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f38a40a3bb0 0x7f38a40a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102a40 0x7f38b819f7c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b8102080 0x7f38b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 --2- 172.21.15.37:0/3786737291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38b8100f30 0x7f38b8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 >> 172.21.15.37:0/3786737291 conn(0x7f38b80fc4a0 msgr2=0x7f38b8105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 shutdown_connections 2024-04-23T18:20:22.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.067+0000 7f38bff44640 1 -- 172.21.15.37:0/3786737291 wait complete. 2024-04-23T18:20:22.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:20:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 -- 172.21.15.37:0/1245941417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 msgr2=0x7f6c5c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/1245941417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c104e20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009900 tx=0x7f6c4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 -- 172.21.15.37:0/1245941417 shutdown_connections 2024-04-23T18:20:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/1245941417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c5c107c90 0x7f6c5c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/1245941417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c5c105360 0x7f6c5c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/1245941417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c104e20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 -- 172.21.15.37:0/1245941417 >> 172.21.15.37:0/1245941417 conn(0x7f6c5c0fc440 msgr2=0x7f6c5c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 -- 172.21.15.37:0/1245941417 shutdown_connections 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.181+0000 7f6c61ce1640 1 -- 172.21.15.37:0/1245941417 wait complete. 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 Processor -- start 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 -- start start 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c5c105360 0x7f6c5c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c5c107c90 0x7f6c5c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10daf0 con 0x7f6c5c102a30 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10d970 con 0x7f6c5c107c90 2024-04-23T18:20:22.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.182+0000 7f6c61ce1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10dc70 con 0x7f6c5c105360 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c1023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47800/0 (socket says 172.21.15.37:47800) 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/681204588 learned_addr learned my addr 172.21.15.37:0/681204588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5affd640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c5c105360 0x7f6c5c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c5c107c90 0x7f6c5c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/681204588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c5c105360 msgr2=0x7f6c5c1009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c5c105360 0x7f6c5c1009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/681204588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c5c107c90 msgr2=0x7f6c5c100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c5c107c90 0x7f6c5c100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/681204588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c1017b0 con 0x7f6c5c102a30 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c1023a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005940 tx=0x7f6c4c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c58ff9640 1 -- 172.21.15.37:0/681204588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c03f030 con 0x7f6c5c102a30 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c58ff9640 1 -- 172.21.15.37:0/681204588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4c003cb0 con 0x7f6c5c102a30 2024-04-23T18:20:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c1a2290 con 0x7f6c5c102a30 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c58ff9640 1 -- 172.21.15.37:0/681204588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c035070 con 0x7f6c5c102a30 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.183+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c1a2830 con 0x7f6c5c102a30 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c58ff9640 1 -- 172.21.15.37:0/681204588 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c4c042030 con 0x7f6c5c102a30 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 Processor -- start 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 -- start start 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a3710 0x55ab3a6a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a606c50 con 0x55ab3a6a2d50 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a5a3830 con 0x55ab3a6a3710 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c649fa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a602c10 con 0x55ab3a6a1b80 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6a31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47808/0 (socket says 172.21.15.37:47808) 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/3947606270 learned_addr learned my addr 172.21.15.37:0/3947606270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5affd640 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a3710 0x55ab3a6a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/3947606270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 msgr2=0x55ab3a6a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/3947606270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a3710 msgr2=0x55ab3a6a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a3710 0x55ab3a6a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.186+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/3947606270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3a3192d0 con 0x55ab3a6a2d50 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c58ff9640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6c440a3c30 0x7f6c440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6c440a3c30 0x7f6c440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6a31d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0f4eb0 tx=0x7f6c4c0f7410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d82b0d3086ccfc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c58ff9640 1 -- 172.21.15.37:0/681204588 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6c4c0f0d90 con 0x7f6c5c102a30 2024-04-23T18:20:22.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c41ffb640 1 -- 172.21.15.37:0/3947606270 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c049070 con 0x55ab3a6a2d50 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c41ffb640 1 -- 172.21.15.37:0/3947606270 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4c0bd760 con 0x55ab3a6a2d50 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.187+0000 7f6c41ffb640 1 -- 172.21.15.37:0/3947606270 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0bce30 con 0x55ab3a6a2d50 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 -- 172.21.15.37:0/3947606270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 msgr2=0x55ab3a6a31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6c440a3c30 0x7f6c440a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ab3a6a2c50 tx=0x7f6c540094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6a31d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0f4eb0 tx=0x7f6c4c0f7410 comp rx=0 tx=0).stop 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 -- 172.21.15.37:0/3947606270 shutdown_connections 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a3710 0x55ab3a6a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6a31d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 --2- 172.21.15.37:0/3947606270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 -- 172.21.15.37:0/3947606270 >> 172.21.15.37:0/3947606270 conn(0x55ab3a69d070 msgr2=0x55ab3a69d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 -- 172.21.15.37:0/3947606270 shutdown_connections 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 -- 172.21.15.37:0/3947606270 wait complete. 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 -- start start 2024-04-23T18:20:22.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 Processor -- start 2024-04-23T18:20:22.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.188+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6aa4e0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a5a1470 con 0x55ab3a6a2d50 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6aa660 con 0x55ab3a6a3710 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 10 client.0 _ll_register_callbacks cb 0x55ab3a5c6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6abd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47822/0 (socket says 172.21.15.37:47822) 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/1562914643 learned_addr learned my addr 172.21.15.37:0/1562914643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5affd640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 msgr2=0x55ab3a6b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/1562914643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 msgr2=0x55ab3a6ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/1562914643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3a3e36b0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6abd80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0b9a30 tx=0x7f6c4c005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c2129e81b6c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c3affd640 1 -- 172.21.15.37:0/1562914643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c049040 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c3affd640 1 -- 172.21.15.37:0/1562914643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6c4c0c10e0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4c0bd760 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c3affd640 1 -- 172.21.15.37:0/1562914643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6c4c0bc9c0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 10 client.15222 fetch_fsmap learned FSMap version 22 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.189+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ab3a6aa4e0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c3affd640 1 -- 172.21.15.37:0/1562914643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4c0bd760 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c3affd640 1 -- 172.21.15.37:0/1562914643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6c4c0b96d0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c649fa740 10 client.15222 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c649fa740 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:20:22.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c649fa740 20 client.15222 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c649fa740 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a6b8a60 0x55ab3a6b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c649fa740 4 client.15222 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab3a614670 con 0x55ab3a6b8a60 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.190+0000 7f6c5affd640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a6b8a60 0x55ab3a6b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c3affd640 10 client.15222 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c5affd640 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a6b8a60 0x55ab3a6b9ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc3afc3ab8223a63 server_cookie=1dd51c86ad33b447 in_seq=0 out_seq=0 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c3affd640 1 -- 172.21.15.37:0/1562914643 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab3a614670 con 0x55ab3a6b8a60 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c3affd640 10 client.15222 handle_command_reply: tid=0 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c649fa740 1 client.15222 shutdown 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 msgr2=0x55ab3a6abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c649fa740 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6abd80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0b9a30 tx=0x7f6c4c005e10 comp rx=0 tx=0).stop 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 shutdown_connections 2024-04-23T18:20:22.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.191+0000 7f6c649fa740 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a6b8a60 0x55ab3a6b9ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 1 --2- 172.21.15.37:0/1562914643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6abd80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c3affd640 0 client.15222 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 >> 172.21.15.37:0/1562914643 conn(0x55ab3a69d070 msgr2=0x55ab3a69c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 shutdown_connections 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 1 -- 172.21.15.37:0/1562914643 wait complete. 2024-04-23T18:20:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.192+0000 7f6c649fa740 20 client.15222 trim_cache size 0 max 16384 2024-04-23T18:20:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 Processor -- start 2024-04-23T18:20:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 -- start start 2024-04-23T18:20:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 0x55ab3a6ab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6b8210 con 0x55ab3a6a3710 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6b88e0 con 0x55ab3a6a2d50 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c649fa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6b8fb0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 0x55ab3a6ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 0x55ab3a6ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47836/0 (socket says 172.21.15.37:47836) 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 -- 172.21.15.37:0/162546888 learned_addr learned my addr 172.21.15.37:0/162546888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 -- 172.21.15.37:0/162546888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 msgr2=0x55ab3a6980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 -- 172.21.15.37:0/162546888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 msgr2=0x55ab3a6b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5bfff640 1 -- 172.21.15.37:0/162546888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3a69dd40 con 0x55ab3a6a3710 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.210+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 0x55ab3a6ab990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6c540127c0 tx=0x7f6c54014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99ff2d1e18061e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c417fa640 1 -- 172.21.15.37:0/162546888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5401e8b0 con 0x55ab3a6a3710 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c417fa640 1 -- 172.21.15.37:0/162546888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c54029430 con 0x55ab3a6a3710 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c417fa640 1 -- 172.21.15.37:0/162546888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5401f680 con 0x55ab3a6a3710 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 -- 172.21.15.37:0/162546888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 msgr2=0x55ab3a6ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 0x55ab3a6ab990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6c540127c0 tx=0x7f6c54014000 comp rx=0 tx=0).stop 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 -- 172.21.15.37:0/162546888 shutdown_connections 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a3710 0x55ab3a6ab990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 --2- 172.21.15.37:0/162546888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 -- 172.21.15.37:0/162546888 >> 172.21.15.37:0/162546888 conn(0x55ab3a69d070 msgr2=0x55ab3a69d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 -- 172.21.15.37:0/162546888 shutdown_connections 2024-04-23T18:20:22.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.211+0000 7f6c649fa740 1 -- 172.21.15.37:0/162546888 wait complete. 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 -- start start 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 Processor -- start 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6b9d50 con 0x55ab3a6a1b80 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6b9680 con 0x55ab3a6a2d50 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ab3a6ba420 con 0x55ab3a6a3710 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c649fa740 10 client.0 _ll_register_callbacks cb 0x7f6c5000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47848/0 (socket says 172.21.15.37:47848) 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/264112054 learned_addr learned my addr 172.21.15.37:0/264112054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5bfff640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5affd640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 msgr2=0x55ab3a6b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/264112054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 msgr2=0x55ab3a6af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5b7fe640 1 -- 172.21.15.37:0/264112054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3a6b2690 con 0x55ab3a6a1b80 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.212+0000 7f6c5affd640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c5b7fe640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6aeb40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0c5e00 tx=0x7f6c4c0c5cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9947197830986f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c3affd640 1 -- 172.21.15.37:0/264112054 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c049070 con 0x55ab3a6a1b80 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c3affd640 1 -- 172.21.15.37:0/264112054 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6c4c03bd00 con 0x55ab3a6a1b80 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c54029430 con 0x55ab3a6a1b80 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c3affd640 1 -- 172.21.15.37:0/264112054 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6c4c0d5070 con 0x55ab3a6a1b80 2024-04-23T18:20:22.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 10 client.15228 fetch_fsmap learned FSMap version 22 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ab3a6b2690 con 0x55ab3a6a1b80 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c3affd640 1 -- 172.21.15.37:0/264112054 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c54029430 con 0x55ab3a6a1b80 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c3affd640 1 -- 172.21.15.37:0/264112054 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6c4c0b96d0 con 0x55ab3a6a1b80 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 10 client.15228 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 10 client.15228 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 20 client.15228 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a723320 0x55ab3a722030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 4 client.15228 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.213+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "status"}) -- 0x55ab3a60e4e0 con 0x55ab3a723320 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.214+0000 7f6c5affd640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a723320 0x55ab3a722030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.214+0000 7f6c3affd640 10 client.15228 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.214+0000 7f6c5affd640 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a723320 0x55ab3a722030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b30491ed161c691 server_cookie=4171a6ce93372ceb in_seq=0 out_seq=0 2024-04-23T18:20:22.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.214+0000 7f6c3affd640 1 -- 172.21.15.37:0/264112054 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ab3a60e4e0 con 0x55ab3a723320 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.214+0000 7f6c3affd640 10 client.15228 handle_command_reply: tid=0 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.214+0000 7f6c649fa740 1 client.15228 shutdown 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 msgr2=0x55ab3a6aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6aeb40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0c5e00 tx=0x7f6c4c0c5cd0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 shutdown_connections 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x55ab3a723320 0x55ab3a722030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ab3a6a3710 0x55ab3a6b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ab3a6a2d50 0x55ab3a6af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 --2- 172.21.15.37:0/264112054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab3a6a1b80 0x55ab3a6aeb40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c3affd640 0 client.15228 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 >> 172.21.15.37:0/264112054 conn(0x55ab3a69d070 msgr2=0x55ab3a6980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 shutdown_connections 2024-04-23T18:20:22.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 1 -- 172.21.15.37:0/264112054 wait complete. 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c649fa740 20 client.15228 trim_cache size 0 max 16384 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24484, 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 104.150810699, 2024-04-23T18:20:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-04-23T18:20:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-23T18:20:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:20:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 121.968184139 2024-04-23T18:20:22.217 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:22.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6c440a3c30 msgr2=0x7f6c440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6c440a3c30 0x7f6c440a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ab3a6a2c50 tx=0x7f6c540094a0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 msgr2=0x7f6c5c1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c1023a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005940 tx=0x7f6c4c004430 comp rx=0 tx=0).stop 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 shutdown_connections 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f6c440a3c30 0x7f6c440a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c5c107c90 0x7f6c5c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c5c105360 0x7f6c5c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 --2- 172.21.15.37:0/681204588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c5c102a30 0x7f6c5c1023a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 >> 172.21.15.37:0/681204588 conn(0x7f6c5c0fc440 msgr2=0x7f6c5c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 shutdown_connections 2024-04-23T18:20:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.215+0000 7f6c61ce1640 1 -- 172.21.15.37:0/681204588 wait complete. 2024-04-23T18:20:22.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:22 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2428510466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:22.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:22 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2578918195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:22.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:22 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1055783664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:22.221 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:20:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 -- 172.21.15.37:0/4114381441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648100f30 msgr2=0x7fd648101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4114381441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648100f30 0x7fd648101350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd638009900 tx=0x7fd63802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 -- 172.21.15.37:0/4114381441 shutdown_connections 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4114381441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd648102a40 0x7fd648109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4114381441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 0x7fd648102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4114381441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648100f30 0x7fd648101350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 -- 172.21.15.37:0/4114381441 >> 172.21.15.37:0/4114381441 conn(0x7fd6480fc4a0 msgr2=0x7fd6480fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 -- 172.21.15.37:0/4114381441 shutdown_connections 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 -- 172.21.15.37:0/4114381441 wait complete. 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 Processor -- start 2024-04-23T18:20:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.329+0000 7fd64e99b640 1 -- start start 2024-04-23T18:20:22.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64e99b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd648100f30 0x7fd648198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64e99b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 0x7fd6481987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64e99b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 0x7fd64819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64e99b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd64810dbf0 con 0x7fd648102a40 2024-04-23T18:20:22.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64e99b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd64810da70 con 0x7fd648100f30 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64e99b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd64810dd70 con 0x7fd648102080 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 0x7fd64819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 0x7fd64819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47856/0 (socket says 172.21.15.37:47856) 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 -- 172.21.15.37:0/4273409464 learned_addr learned my addr 172.21.15.37:0/4273409464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd6477fe640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 0x7fd6481987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd647fff640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd648100f30 0x7fd648198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 -- 172.21.15.37:0/4273409464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 msgr2=0x7fd6481987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 0x7fd6481987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 -- 172.21.15.37:0/4273409464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd648100f30 msgr2=0x7fd648198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd648100f30 0x7fd648198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 -- 172.21.15.37:0/4273409464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64819fe80 con 0x7fd648102a40 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd6477fe640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 0x7fd6481987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd64cf38640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 0x7fd64819f810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd64000cce0 tx=0x7fd64000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.330+0000 7fd6457fa640 1 -- 172.21.15.37:0/4273409464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64000f450 con 0x7fd648102a40 2024-04-23T18:20:22.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.331+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6481a0170 con 0x7fd648102a40 2024-04-23T18:20:22.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.331+0000 7fd6457fa640 1 -- 172.21.15.37:0/4273409464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd64000faa0 con 0x7fd648102a40 2024-04-23T18:20:22.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.331+0000 7fd6457fa640 1 -- 172.21.15.37:0/4273409464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640018b70 con 0x7fd648102a40 2024-04-23T18:20:22.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.331+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6481a06c0 con 0x7fd648102a40 2024-04-23T18:20:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd6457fa640 1 -- 172.21.15.37:0/4273409464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd640018d10 con 0x7fd648102a40 2024-04-23T18:20:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 Processor -- start 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 -- start start 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ccf90 0x559d138cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559d1382dc50 con 0x559d138ceb20 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559d137ca830 con 0x559d138ccf90 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd651793740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559d13829c10 con 0x559d138ce160 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47864/0 (socket says 172.21.15.37:47864) 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd6477fe640 1 -- 172.21.15.37:0/470999673 learned_addr learned my addr 172.21.15.37:0/470999673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd6477fe640 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ccf90 0x559d138cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd6457fa640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6300a3bb0 0x7fd6300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.333+0000 7fd647fff640 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd64cf38640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6300a3bb0 0x7fd6300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6477fe640 1 -- 172.21.15.37:0/470999673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 msgr2=0x559d138ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6477fe640 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6477fe640 1 -- 172.21.15.37:0/470999673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ccf90 msgr2=0x559d138cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6477fe640 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ccf90 0x559d138cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6477fe640 1 -- 172.21.15.37:0/470999673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d135402d0 con 0x559d138ceb20 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6457fa640 1 -- 172.21.15.37:0/4273409464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd6400d08b0 con 0x7fd648102a40 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd6477fe640 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d53b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd63c014a00 tx=0x7fd63c00b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6fb1aa1e3741c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd62e7fc640 1 -- 172.21.15.37:0/470999673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00f820 con 0x559d138ceb20 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd64cf38640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6300a3bb0 0x7fd6300a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd64009c880 tx=0x7fd6400671d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd62e7fc640 1 -- 172.21.15.37:0/470999673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c00fe70 con 0x559d138ceb20 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.334+0000 7fd62e7fc640 1 -- 172.21.15.37:0/470999673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c012ac0 con 0x559d138ceb20 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 -- 172.21.15.37:0/470999673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 msgr2=0x559d138d53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d53b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd63c014a00 tx=0x7fd63c00b550 comp rx=0 tx=0).stop 2024-04-23T18:20:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 -- 172.21.15.37:0/470999673 shutdown_connections 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d53b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 --2- 172.21.15.37:0/470999673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ccf90 0x559d138cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 -- 172.21.15.37:0/470999673 >> 172.21.15.37:0/470999673 conn(0x559d138c4070 msgr2=0x559d138c4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 -- 172.21.15.37:0/470999673 shutdown_connections 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 -- 172.21.15.37:0/470999673 wait complete. 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 -- start start 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.335+0000 7fd651793740 1 Processor -- start 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138d7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ceb20 0x559d138de9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559d138d5b30 con 0x559d138ccf90 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559d137c8470 con 0x559d138ceb20 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559d138d5cb0 con 0x559d138ce160 2024-04-23T18:20:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 10 client.0 _ll_register_callbacks cb 0x559d137edf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138d7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47868/0 (socket says 172.21.15.37:47868) 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 -- 172.21.15.37:0/4064958237 learned_addr learned my addr 172.21.15.37:0/4064958237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd647fff640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ceb20 0x559d138de9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 -- 172.21.15.37:0/4064958237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 msgr2=0x559d138d7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138d7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 -- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ceb20 msgr2=0x559d138de9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ceb20 0x559d138de9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 -- 172.21.15.37:0/4064958237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1360a6b0 con 0x559d138ccf90 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6477fe640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138d7420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd63c005890 tx=0x7fd63c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c41e3be42986a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6277fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6277fe640 1 -- 172.21.15.37:0/4064958237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c012450 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6277fe640 1 -- 172.21.15.37:0/4064958237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd63c005250 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63c00fe70 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd6277fe640 1 -- 172.21.15.37:0/4064958237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd63c005450 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 10 client.15240 fetch_fsmap learned FSMap version 22 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.336+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559d138e1d10 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd6277fe640 1 -- 172.21.15.37:0/4064958237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c00fe70 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd6277fe640 1 -- 172.21.15.37:0/4064958237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd63c02b020 con 0x559d138ccf90 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd651793740 10 client.15240 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd651793740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd651793740 20 client.15240 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd651793740 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d138e4080 0x559d138e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd651793740 4 client.15240 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d1383b670 con 0x559d138e4080 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.337+0000 7fd647fff640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d138e4080 0x559d138e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.338+0000 7fd647fff640 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d138e4080 0x559d138e54f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a79679d471ff606 server_cookie=a18d704d935cd1ce in_seq=0 out_seq=0 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.338+0000 7fd6277fe640 10 client.15240 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.338+0000 7fd6277fe640 1 -- 172.21.15.37:0/4064958237 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d1383b670 con 0x559d138e4080 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.338+0000 7fd6277fe640 10 client.15240 handle_command_reply: tid=0 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 client.15240 shutdown 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 msgr2=0x559d138d7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138d7420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd63c005890 tx=0x7fd63c001e70 comp rx=0 tx=0).stop 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 shutdown_connections 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d138e4080 0x559d138e54f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ceb20 0x559d138de9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ce160 0x559d138d7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd6277fe640 0 client.15240 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 --2- 172.21.15.37:0/4064958237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138d7420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 >> 172.21.15.37:0/4064958237 conn(0x559d138c4070 msgr2=0x559d138c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 shutdown_connections 2024-04-23T18:20:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 1 -- 172.21.15.37:0/4064958237 wait complete. 2024-04-23T18:20:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.339+0000 7fd651793740 20 client.15240 trim_cache size 0 max 16384 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.357+0000 7fd651793740 1 Processor -- start 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.357+0000 7fd651793740 1 -- start start 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd651793740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ccf90 0x559d138bf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd651793740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd651793740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd651793740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559d138e3830 con 0x559d138ceb20 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd651793740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559d138e3f00 con 0x559d138ce160 2024-04-23T18:20:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd651793740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559d138e45d0 con 0x559d138ccf90 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47878/0 (socket says 172.21.15.37:47878) 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 -- 172.21.15.37:0/141907218 learned_addr learned my addr 172.21.15.37:0/141907218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd647fff640 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ccf90 0x559d138bf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 -- 172.21.15.37:0/141907218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ccf90 msgr2=0x559d138bf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ccf90 0x559d138bf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 -- 172.21.15.37:0/141907218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 msgr2=0x559d138de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 -- 172.21.15.37:0/141907218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d138d8410 con 0x559d138ceb20 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd647fff640 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd6477fe640 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d7030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00f730 tx=0x7fd63c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaba2cc11edd0db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd62dffb640 1 -- 172.21.15.37:0/141907218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00bc50 con 0x559d138ceb20 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd62dffb640 1 -- 172.21.15.37:0/141907218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c014520 con 0x559d138ceb20 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.358+0000 7fd62dffb640 1 -- 172.21.15.37:0/141907218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c020930 con 0x559d138ceb20 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 -- 172.21.15.37:0/141907218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 msgr2=0x559d138d7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d7030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00f730 tx=0x7fd63c000bc0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 -- 172.21.15.37:0/141907218 shutdown_connections 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ceb20 0x559d138d7030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 --2- 172.21.15.37:0/141907218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ccf90 0x559d138bf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 -- 172.21.15.37:0/141907218 >> 172.21.15.37:0/141907218 conn(0x559d138c4070 msgr2=0x559d138c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 -- 172.21.15.37:0/141907218 shutdown_connections 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 -- 172.21.15.37:0/141907218 wait complete. 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 -- start start 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.359+0000 7fd651793740 1 Processor -- start 2024-04-23T18:20:22.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ceb20 0x559d138dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559d138e5370 con 0x559d138ccf90 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559d138e4ca0 con 0x559d138ce160 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559d138e5a40 con 0x559d138ceb20 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 10 client.0 _ll_register_callbacks cb 0x7fd63c014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138da120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47884/0 (socket says 172.21.15.37:47884) 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 -- 172.21.15.37:0/1337104059 learned_addr learned my addr 172.21.15.37:0/1337104059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd647fff640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ceb20 0x559d138dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 -- 172.21.15.37:0/1337104059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ceb20 msgr2=0x559d138dd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ceb20 0x559d138dd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 -- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 msgr2=0x559d138da660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138da660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 -- 172.21.15.37:0/1337104059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d138ddc70 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6477fe640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138da120 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd63c014360 tx=0x7fd63c00fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07cf0f3336d6596 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6277fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6277fe640 1 -- 172.21.15.37:0/1337104059 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c03e020 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd6277fe640 1 -- 172.21.15.37:0/1337104059 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fd63c005250 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.360+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63c014520 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd6277fe640 1 -- 172.21.15.37:0/1337104059 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd63c004ac0 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 10 client.15246 fetch_fsmap learned FSMap version 22 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559d1394a0f0 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd6277fe640 1 -- 172.21.15.37:0/1337104059 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c014520 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd6277fe640 1 -- 172.21.15.37:0/1337104059 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd63c02b040 con 0x559d138ccf90 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 10 client.15246 fetch_fsmap finished waiting for FSMap version 22 2024-04-23T18:20:22.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 20 client.15246 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d1394eab0 0x559d1394d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 4 client.15246 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "status"}) -- 0x559d138354e0 con 0x559d1394eab0 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.361+0000 7fd647fff640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d1394eab0 0x559d1394d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd647fff640 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d1394eab0 0x559d1394d7c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb0e4248fe4b708 server_cookie=ccaf5dab67a33885 in_seq=0 out_seq=0 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd6277fe640 10 client.15246 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd6277fe640 1 -- 172.21.15.37:0/1337104059 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559d138354e0 con 0x559d1394eab0 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd6277fe640 10 client.15246 handle_command_reply: tid=0 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 client.15246 shutdown 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 msgr2=0x559d138da120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138da120 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd63c014360 tx=0x7fd63c00fc50 comp rx=0 tx=0).stop 2024-04-23T18:20:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 shutdown_connections 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x559d1394eab0 0x559d1394d7c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559d138ceb20 0x559d138dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd6277fe640 0 client.15246 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d138ce160 0x559d138da660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 --2- 172.21.15.37:0/1337104059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d138ccf90 0x559d138da120 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 >> 172.21.15.37:0/1337104059 conn(0x559d138c4070 msgr2=0x559d138c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 shutdown_connections 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 1 -- 172.21.15.37:0/1337104059 wait complete. 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.362+0000 7fd651793740 20 client.15246 trim_cache size 0 max 16384 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:22.363 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24509, 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 107.327169911, 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 22, 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 113.831388175 2024-04-23T18:20:22.364 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6300a3bb0 msgr2=0x7fd6300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6300a3bb0 0x7fd6300a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd64009c880 tx=0x7fd6400671d0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 msgr2=0x7fd64819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 0x7fd64819f810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd64000cce0 tx=0x7fd64000a620 comp rx=0 tx=0).stop 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 shutdown_connections 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fd6300a3bb0 0x7fd6300a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd648102a40 0x7fd64819f810 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd648102080 0x7fd6481987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 --2- 172.21.15.37:0/4273409464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd648100f30 0x7fd648198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 >> 172.21.15.37:0/4273409464 conn(0x7fd6480fc4a0 msgr2=0x7fd6481053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 shutdown_connections 2024-04-23T18:20:22.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.363+0000 7fd64e99b640 1 -- 172.21.15.37:0/4273409464 wait complete. 2024-04-23T18:20:22.368 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T18:20:22.378 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T18:20:22.378 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T18:20:22.378 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T18:20:22.378 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-23T18:20:22.381 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:20:22.381 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:20:22.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-23T18:20:22.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 -- 172.21.15.37:0/2994536026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 msgr2=0x7f5c9c106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/2994536026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c106070 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009900 tx=0x7f5c9402a410 comp rx=0 tx=0).stop 2024-04-23T18:20:22.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 -- 172.21.15.37:0/2994536026 shutdown_connections 2024-04-23T18:20:22.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/2994536026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c106070 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/2994536026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c9c103240 0x7f5c9c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/2994536026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c9c1008d0 0x7f5c9c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 -- 172.21.15.37:0/2994536026 >> 172.21.15.37:0/2994536026 conn(0x7f5c9c0fc4d0 msgr2=0x7f5c9c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 -- 172.21.15.37:0/2994536026 shutdown_connections 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 -- 172.21.15.37:0/2994536026 wait complete. 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 Processor -- start 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.489+0000 7f5ca39f9640 1 -- start start 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca39f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c9c1008d0 0x7f5c9c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca39f9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c9c103240 0x7f5c9c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca39f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c1a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca39f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c114100 con 0x7f5c9c105bf0 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca39f9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c113f80 con 0x7f5c9c103240 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca39f9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c114280 con 0x7f5c9c1008d0 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c1a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c1a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47916/0 (socket says 172.21.15.37:47916) 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 -- 172.21.15.37:0/3614430868 learned_addr learned my addr 172.21.15.37:0/3614430868 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca0f94640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c9c103240 0x7f5c9c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1795640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c9c1008d0 0x7f5c9c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 -- 172.21.15.37:0/3614430868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c9c1008d0 msgr2=0x7f5c9c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c9c1008d0 0x7f5c9c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 -- 172.21.15.37:0/3614430868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c9c103240 msgr2=0x7f5c9c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c9c103240 0x7f5c9c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 -- 172.21.15.37:0/3614430868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c1a6230 con 0x7f5c9c105bf0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5ca1f96640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c1a5bc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c94031500 tx=0x7f5c94031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9403f030 con 0x7f5c9c105bf0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c94003aa0 con 0x7f5c9c105bf0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.490+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94035070 con 0x7f5c9c105bf0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.491+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c1a64c0 con 0x7f5c9c105bf0 2024-04-23T18:20:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.491+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1a6960 con 0x7f5c9c105bf0 2024-04-23T18:20:22.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.492+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c0fdfc0 con 0x7f5c9c105bf0 2024-04-23T18:20:22.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.493+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5c94042030 con 0x7f5c9c105bf0 2024-04-23T18:20:22.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.493+0000 7f5c9a7fc640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c840a3c50 0x7f5c840a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:22.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.493+0000 7f5ca1795640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c840a3c50 0x7f5c840a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:22.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.493+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f5c940f1020 con 0x7f5c9c105bf0 2024-04-23T18:20:22.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.494+0000 7f5ca1795640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c840a3c50 0x7f5c840a6110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007350 tx=0x7f5c8c0072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:22.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.497+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c940bd9f0 con 0x7f5c9c105bf0 2024-04-23T18:20:22.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:22 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2428510466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:22.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:22 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2578918195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:22.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:22 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1055783664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:22.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:22 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2428510466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:22.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:22 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2578918195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:22.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:22 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1055783664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:22.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:22.738+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f5c9c062b50 con 0x7f5c9c105bf0 2024-04-23T18:20:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.161+0000 7f5c9a7fc640 1 -- 172.21.15.37:0/3614430868 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f5c940bd2f0 con 0x7f5c9c105bf0 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c840a3c50 msgr2=0x7f5c840a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c840a3c50 0x7f5c840a6110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007350 tx=0x7f5c8c0072e0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 msgr2=0x7f5c9c1a5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c1a5bc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c94031500 tx=0x7f5c94031a50 comp rx=0 tx=0).stop 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 shutdown_connections 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f5c840a3c50 0x7f5c840a6110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c9c105bf0 0x7f5c9c1a5bc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c9c103240 0x7f5c9c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 --2- 172.21.15.37:0/3614430868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c9c1008d0 0x7f5c9c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 >> 172.21.15.37:0/3614430868 conn(0x7f5c9c0fc4d0 msgr2=0x7f5c9c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 shutdown_connections 2024-04-23T18:20:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.165+0000 7f5ca39f9640 1 -- 172.21.15.37:0/3614430868 wait complete. 2024-04-23T18:20:23.179 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 -- 172.21.15.37:0/309911796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c105360 msgr2=0x7fef4c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 --2- 172.21.15.37:0/309911796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c105360 0x7fef4c107750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fef38009900 tx=0x7fef3802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 -- 172.21.15.37:0/309911796 shutdown_connections 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 --2- 172.21.15.37:0/309911796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c107c90 0x7fef4c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 --2- 172.21.15.37:0/309911796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c105360 0x7fef4c107750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 --2- 172.21.15.37:0/309911796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef4c102a30 0x7fef4c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.296+0000 7fef530ae640 1 -- 172.21.15.37:0/309911796 >> 172.21.15.37:0/309911796 conn(0x7fef4c0fc420 msgr2=0x7fef4c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 -- 172.21.15.37:0/309911796 shutdown_connections 2024-04-23T18:20:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 -- 172.21.15.37:0/309911796 wait complete. 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 Processor -- start 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 -- start start 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 0x7fef4c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c105360 0x7fef4c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef4c107c90 0x7fef4c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c10daf0 con 0x7fef4c102a30 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c10d970 con 0x7fef4c105360 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef530ae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c10dc70 con 0x7fef4c107c90 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef50e4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 0x7fef4c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef50e4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 0x7fef4c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47932/0 (socket says 172.21.15.37:47932) 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.297+0000 7fef50e4a640 1 -- 172.21.15.37:0/3534225985 learned_addr learned my addr 172.21.15.37:0/3534225985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef4bfff640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c105360 0x7fef4c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef5164b640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef4c107c90 0x7fef4c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef50e4a640 1 -- 172.21.15.37:0/3534225985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef4c107c90 msgr2=0x7fef4c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef50e4a640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef4c107c90 0x7fef4c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef50e4a640 1 -- 172.21.15.37:0/3534225985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c105360 msgr2=0x7fef4c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef50e4a640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c105360 0x7fef4c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef50e4a640 1 -- 172.21.15.37:0/3534225985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c19fdc0 con 0x7fef4c102a30 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef4bfff640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c105360 0x7fef4c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef50e4a640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 0x7fef4c1981a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fef4000e9f0 tx=0x7fef4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4000ccc0 con 0x7fef4c102a30 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef4000ce60 con 0x7fef4c102a30 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1a00b0 con 0x7fef4c102a30 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40010430 con 0x7fef4c102a30 2024-04-23T18:20:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.298+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1a0580 con 0x7fef4c102a30 2024-04-23T18:20:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.300+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c0fdf10 con 0x7fef4c102a30 2024-04-23T18:20:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.301+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fef400046c0 con 0x7fef4c102a30 2024-04-23T18:20:23.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.301+0000 7fef49ffb640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef3c0a3ca0 0x7fef3c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.301+0000 7fef4bfff640 1 -- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef3c0a3ca0 msgr2=0x7fef3c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:23.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.301+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7fef400d0840 con 0x7fef4c102a30 2024-04-23T18:20:23.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.301+0000 7fef4bfff640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef3c0a3ca0 0x7fef3c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:20:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.306+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef4009d210 con 0x7fef4c102a30 2024-04-23T18:20:23.488 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: ignoring --setuser ceph since I am not root 2024-04-23T18:20:23.488 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: ignoring --setgroup ceph since I am not root 2024-04-23T18:20:23.488 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-mgr[33839]: -- 172.21.15.172:0/3891292407 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5011085a0 con 0x55c5010dc800 2024-04-23T18:20:23.489 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:23.345+0000 7fc05c7a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:20:23.489 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-mon[32456]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-23T18:20:23.489 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3614430868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:20:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.495+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fef4c02d5c0 con 0x7fef4c102a30 2024-04-23T18:20:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.499+0000 7fef49ffb640 1 -- 172.21.15.37:0/3534225985 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fef4009cb10 con 0x7fef4c102a30 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef3c0a3ca0 msgr2=0x7fef3c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef3c0a3ca0 0x7fef3c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 msgr2=0x7fef4c1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 0x7fef4c1981a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fef4000e9f0 tx=0x7fef4000edf0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 shutdown_connections 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7fef3c0a3ca0 0x7fef3c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef4c107c90 0x7fef4c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef4c105360 0x7fef4c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 --2- 172.21.15.37:0/3534225985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef4c102a30 0x7fef4c1981a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 >> 172.21.15.37:0/3534225985 conn(0x7fef4c0fc420 msgr2=0x7fef4c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 shutdown_connections 2024-04-23T18:20:23.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.501+0000 7fef530ae640 1 -- 172.21.15.37:0/3534225985 wait complete. 2024-04-23T18:20:23.506 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setuser ceph since I am not root 2024-04-23T18:20:23.506 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: ignoring --setgroup ceph since I am not root 2024-04-23T18:20:23.506 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:23.357+0000 7fbbb9e06200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:20:23.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-mon[29404]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-23T18:20:23.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3614430868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:20:23.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-23T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-mon[32906]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-23T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3614430868' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:20:23.525 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: ignoring --setuser ceph since I am not root 2024-04-23T18:20:23.525 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: ignoring --setgroup ceph since I am not root 2024-04-23T18:20:23.525 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-mgr[34773]: -- 172.21.15.173:0/1481557129 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a342765a0 con 0x561a3424a800 2024-04-23T18:20:23.525 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:23.347+0000 7fdeeb065200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:20:23.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 -- 172.21.15.37:0/3076973184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102a00 msgr2=0x7ff664109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 --2- 172.21.15.37:0/3076973184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102a00 0x7ff664109320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009be0 tx=0x7ff65c02a6c0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 -- 172.21.15.37:0/3076973184 shutdown_connections 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 --2- 172.21.15.37:0/3076973184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102a00 0x7ff664109320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 --2- 172.21.15.37:0/3076973184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff664102040 0x7ff6641024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 --2- 172.21.15.37:0/3076973184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 -- 172.21.15.37:0/3076973184 >> 172.21.15.37:0/3076973184 conn(0x7ff6640fc420 msgr2=0x7ff6640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 -- 172.21.15.37:0/3076973184 shutdown_connections 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.617+0000 7ff66c344640 1 -- 172.21.15.37:0/3076973184 wait complete. 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 Processor -- start 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 -- start start 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102040 0x7ff6641034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff664102a00 0x7ff6641039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410ca00 con 0x7ff664100e60 2024-04-23T18:20:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410c880 con 0x7ff664102a00 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66c344640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410cb80 con 0x7ff664102040 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66a0e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66a0e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47948/0 (socket says 172.21.15.37:47948) 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66a0e0640 1 -- 172.21.15.37:0/2178693878 learned_addr learned my addr 172.21.15.37:0/2178693878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff6698df640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102040 0x7ff6641034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66a8e1640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff664102a00 0x7ff6641039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff6698df640 1 -- 172.21.15.37:0/2178693878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff664102a00 msgr2=0x7ff6641039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff6698df640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff664102a00 0x7ff6641039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff6698df640 1 -- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 msgr2=0x7ff664104e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff6698df640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664104e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff6698df640 1 -- 172.21.15.37:0/2178693878 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff664104280 con 0x7ff664102040 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.618+0000 7ff66a0e0640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.619+0000 7ff6698df640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102040 0x7ff6641034b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff66000e9f0 tx=0x7ff66000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.619+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66000ccc0 con 0x7ff664102040 2024-04-23T18:20:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.619+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff660005020 con 0x7ff664102040 2024-04-23T18:20:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.619+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641a2200 con 0x7ff664102040 2024-04-23T18:20:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.619+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660004a00 con 0x7ff664102040 2024-04-23T18:20:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.619+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a2700 con 0x7ff664102040 2024-04-23T18:20:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.620+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6641a2a20 con 0x7ff664102040 2024-04-23T18:20:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.621+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7ff660004ba0 con 0x7ff664102040 2024-04-23T18:20:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.621+0000 7ff65b7fe640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff6500a3ca0 0x7ff6500a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.621+0000 7ff66a0e0640 1 -- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff6500a3ca0 msgr2=0x7ff6500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.622+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7ff6600d0660 con 0x7ff664102040 2024-04-23T18:20:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.622+0000 7ff66a0e0640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff6500a3ca0 0x7ff6500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:20:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.625+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff660042e20 con 0x7ff664102040 2024-04-23T18:20:23.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.813+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff6641a2e20 con 0x7ff664102040 2024-04-23T18:20:23.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.818+0000 7ff65b7fe640 1 -- 172.21.15.37:0/2178693878 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7ff66009cfd0 con 0x7ff664102040 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff6500a3ca0 msgr2=0x7ff6500a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff6500a3ca0 0x7ff6500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102040 msgr2=0x7ff6641034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102040 0x7ff6641034b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff66000e9f0 tx=0x7ff66000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 shutdown_connections 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7ff6500a3ca0 0x7ff6500a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff664102a00 0x7ff6641039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.820+0000 7ff66c344640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff664102040 0x7ff6641034b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.821+0000 7ff66c344640 1 --2- 172.21.15.37:0/2178693878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664100e60 0x7ff664104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:23.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.821+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 >> 172.21.15.37:0/2178693878 conn(0x7ff6640fc420 msgr2=0x7ff6640fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:23.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.821+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 shutdown_connections 2024-04-23T18:20:23.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:23.821+0000 7ff66c344640 1 -- 172.21.15.37:0/2178693878 wait complete. 2024-04-23T18:20:23.825 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:23.553+0000 7fbbb9e06200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:20:23.825 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:23.663+0000 7fbbb9e06200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:20:23.826 INFO:teuthology.run_tasks:Running task kclient... 2024-04-23T18:20:23.849 INFO:tasks.kclient:Mounting kernel clients... 2024-04-23T18:20:23.849 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-23T18:20:23.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:23.850 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-23T18:20:23.850 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:23.529+0000 7fc05c7a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:20:23.851 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:23.632+0000 7fc05c7a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:20:23.855 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:23.531+0000 7fdeeb065200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:20:23.855 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:23.635+0000 7fdeeb065200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:20:23.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:23.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-23T18:20:23.933 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-23T18:20:23.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:20:23.934 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-23T18:20:23.935 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T18:20:23.935 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:23.935 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T18:20:24.076 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:23 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:23.990+0000 7fbbb9e06200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:20:24.238 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:24.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:24 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3614430868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:20:24.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:24 smithi172 ceph-mon[32456]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-23T18:20:24.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:24 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3534225985' entity='client.admin' 2024-04-23T18:20:24.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:24 smithi172 ceph-mon[32456]: from='client.? ' entity='client.admin' 2024-04-23T18:20:24.338 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:23.943+0000 7fc05c7a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:20:24.338 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:24.028+0000 7fc05c7a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:20:24.355 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:23.943+0000 7fdeeb065200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:20:24.355 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:24.028+0000 7fdeeb065200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:20:24.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:24 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3614430868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:20:24.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:24 smithi173 ceph-mon[32906]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-23T18:20:24.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:24 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3534225985' entity='client.admin' 2024-04-23T18:20:24.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:24 smithi173 ceph-mon[32906]: from='client.? ' entity='client.admin' 2024-04-23T18:20:24.360 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:24 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:24.081+0000 7fbbb9e06200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:20:24.360 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:24 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3614430868' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:20:24.360 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:24 smithi037 ceph-mon[29404]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-23T18:20:24.360 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:24 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3534225985' entity='client.admin' 2024-04-23T18:20:24.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:24 smithi037 ceph-mon[29404]: from='client.? ' entity='client.admin' 2024-04-23T18:20:25.542 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:25.207+0000 7fc05c7a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:20:25.542 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:25.378+0000 7fc05c7a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:20:25.542 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:25.207+0000 7fdeeb065200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:20:25.543 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:25.377+0000 7fdeeb065200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:20:25.620 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:25 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:25.367+0000 7fbbb9e06200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:20:25.621 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:25 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:25.556+0000 7fbbb9e06200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:20:25.833 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:25.549+0000 7fdeeb065200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:20:25.833 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:25.548+0000 7fc05c7a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:20:25.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:25 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:25.740+0000 7fbbb9e06200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 -- 172.21.15.37:0/706700132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6574107e10 msgr2=0x7f657410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 --2- 172.21.15.37:0/706700132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6574107e10 0x7f657410f990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b0d0 tx=0x7f656c00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 -- 172.21.15.37:0/706700132 shutdown_connections 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 --2- 172.21.15.37:0/706700132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6574107e10 0x7f657410f990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 --2- 172.21.15.37:0/706700132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f657406b370 0x7f657406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 --2- 172.21.15.37:0/706700132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 0x7f65741076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.011+0000 7f657bb78700 1 -- 172.21.15.37:0/706700132 >> 172.21.15.37:0/706700132 conn(0x7f65740788f0 msgr2=0x7f6574078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 -- 172.21.15.37:0/706700132 shutdown_connections 2024-04-23T18:20:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 -- 172.21.15.37:0/706700132 wait complete. 2024-04-23T18:20:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 Processor -- start 2024-04-23T18:20:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 -- start start 2024-04-23T18:20:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 0x7f6574101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 0x7f6574102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.012+0000 7f657bb78700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6574107e10 0x7f6574102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f657bb78700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6574112c30 con 0x7f657406b370 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f657bb78700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6574112ab0 con 0x7f657406bd20 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f657bb78700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6574112db0 con 0x7f6574107e10 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 0x7f6574101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579113700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 0x7f6574102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 0x7f6574101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36368/0 (socket says 172.21.15.37:36368) 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 -- 172.21.15.37:0/814066576 learned_addr learned my addr 172.21.15.37:0/814066576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f657a115700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6574107e10 0x7f6574102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 -- 172.21.15.37:0/814066576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6574107e10 msgr2=0x7f6574102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6574107e10 0x7f6574102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 -- 172.21.15.37:0/814066576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 msgr2=0x7f6574102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 0x7f6574102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 -- 172.21.15.37:0/814066576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741a7570 con 0x7f657406b370 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579113700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 0x7f6574102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f6579914700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 0x7f6574101df0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f657000d950 tx=0x7f657000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570016070 con 0x7f657406b370 2024-04-23T18:20:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741a7860 con 0x7f657406b370 2024-04-23T18:20:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657000fd40 con 0x7f657406b370 2024-04-23T18:20:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657000e3b0 con 0x7f657406b370 2024-04-23T18:20:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.013+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741a7df0 con 0x7f657406b370 2024-04-23T18:20:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.015+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6574104190 con 0x7f657406b370 2024-04-23T18:20:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.016+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6570019070 con 0x7f657406b370 2024-04-23T18:20:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.016+0000 7f656affd700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 0x7f65600a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.016+0000 7f6579113700 1 -- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 msgr2=0x7f65600a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.017+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f65700cff20 con 0x7f657406b370 2024-04-23T18:20:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.017+0000 7f6579113700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 0x7f65600a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:20:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.021+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f657009c8f0 con 0x7f657406b370 2024-04-23T18:20:26.087 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:25.840+0000 7fc05c7a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:20:26.088 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:25.933+0000 7fc05c7a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:20:26.105 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:25.839+0000 7fdeeb065200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:20:26.105 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:25.932+0000 7fdeeb065200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:20:26.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.218+0000 7f6579113700 1 -- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 msgr2=0x7f65600a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:26.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.218+0000 7f6579113700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 0x7f65600a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:20:26.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.279+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f657402e140 con 0x7f657406b370 2024-04-23T18:20:26.273 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:26 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:26.053+0000 7fbbb9e06200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:20:26.273 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:26 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:26.152+0000 7fbbb9e06200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:20:26.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.279+0000 7f656affd700 1 -- 172.21.15.37:0/814066576 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f657009c1f0 con 0x7f657406b370 2024-04-23T18:20:26.274 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 msgr2=0x7f65600a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 0x7f65600a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 msgr2=0x7f6574101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 0x7f6574101df0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f657000d950 tx=0x7f657000dc60 comp rx=0 tx=0).stop 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 shutdown_connections 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f65600a3c90 0x7f65600a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657406b370 0x7f6574101df0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6574107e10 0x7f6574102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 --2- 172.21.15.37:0/814066576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f657406bd20 0x7f6574102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:26.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 >> 172.21.15.37:0/814066576 conn(0x7f65740788f0 msgr2=0x7f65741068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 shutdown_connections 2024-04-23T18:20:26.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:26.282+0000 7f657bb78700 1 -- 172.21.15.37:0/814066576 wait complete. 2024-04-23T18:20:26.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:26 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/814066576' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:20:26.575 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:26 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:26.370+0000 7fbbb9e06200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:20:26.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:26 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/814066576' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:20:26.587 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:26.138+0000 7fc05c7a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:20:26.587 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:26.330+0000 7fc05c7a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:20:26.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:26 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/814066576' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:20:26.605 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:26.138+0000 7fdeeb065200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:20:26.605 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:26.331+0000 7fdeeb065200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:20:26.870 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:26 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:26.580+0000 7fbbb9e06200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:20:26.985 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T18:20:26.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T18:20:26.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-23T18:20:26.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T18:20:26.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:26.986 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T18:20:26.986 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T18:20:26.986 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T18:20:26.986 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:20:26.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:26.986 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:20:27.004 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:20:27.005 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-23T18:20:27.006 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T18:20:27.006 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-23T18:20:27.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:20:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:27.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:27.225 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T18:20:27.252 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-23T18:20:27.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:27.254 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-23T18:20:27.332 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-23T18:20:27.332 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T18:20:27.332 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T18:20:27.332 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T18:20:27.332 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T18:20:27.333 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:20:27.333 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-23T18:20:27.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T18:20:27.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T18:20:27.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T18:20:27.333 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-23T18:20:27.337 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:26.934+0000 7fc05c7a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:20:27.355 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:26.936+0000 7fdeeb065200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:20:27.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:20:27.520 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:27 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:27.270+0000 7fbbb9e06200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:20:27.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:27.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:27.611 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-23T18:20:27.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:27.628 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-23T18:20:27.652 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:27.361+0000 7fc05c7a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:20:27.652 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:27.457+0000 7fc05c7a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:20:27.652 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:27.658+0000 7fc05c7a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:20:27.657 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:27.363+0000 7fdeeb065200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:20:27.657 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:27.460+0000 7fdeeb065200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:20:27.684 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:20:27.684 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-23T18:20:27.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:20:27.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T18:20:27.684 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-23T18:20:27.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:20:27.781 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:27 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:27.730+0000 7fbbb9e06200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:20:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:27.805 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T18:20:27.805 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-23T18:20:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:20:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:27.934 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:20:27.934 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-23T18:20:27.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-23T18:20:27.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T18:20:27.934 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-23T18:20:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:20:28.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:28.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:20:28.013 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:28.029 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:20:28.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:20:28.029 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:28.057 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:27 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:27.836+0000 7fbbb9e06200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:20:28.071 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-23T18:20:28.071 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-23T18:20:28.071 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/d3b18bbc-ac32-4ec7-a1cb-5a6ae034703b 2024-04-23T18:20:28.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:28.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/d3b18bbc-ac32-4ec7-a1cb-5a6ae034703b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-23T18:20:28.105 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:27.664+0000 7fdeeb065200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:20:28.163 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-23T18:20:28.163 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-23T18:20:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-23T18:20:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-23T18:20:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.172:3300,172.21.15.173:3300" 2024-04-23T18:20:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 0@af48a038-019c-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/d3b18bbc-ac32-4ec7-a1cb-5a6ae034703b 2024-04-23T18:20:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.37:3300/172.21.15.172:3300/172.21.15.173:3300" will pass to kernel 2024-04-23T18:20:28.280 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-23T18:20:28.280 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-23T18:20:28.280 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-23T18:20:28.280 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-23T18:20:28.280 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-23T18:20:28.282 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-23T18:20:28.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:20:28.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:28.307 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:28 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:28.063+0000 7fbbb9e06200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:20:28.308 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:20:28.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:20:28.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:28.405 INFO:teuthology.orchestra.run.smithi037.stdout:client24934 2024-04-23T18:20:28.406 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:20:28.406 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:20:28.423 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:20:28.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:20:28.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:28.516 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:28.184+0000 7fc05c7a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:20:28.517 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:28.277+0000 7fc05c7a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:20:28.517 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:28.187+0000 7fdeeb065200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:20:28.517 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:28.281+0000 7fdeeb065200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:20:28.520 INFO:teuthology.orchestra.run.smithi037.stdout:client24934 2024-04-23T18:20:28.520 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:20:28.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:20:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:28.578 INFO:teuthology.orchestra.run.smithi037.stdout:client24934 2024-04-23T18:20:28.579 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af48a038-019c-11ef-bc93-c7b262605968.client24934/status') 2024-04-23T18:20:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:20:28.638 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24934 (3)172.21.15.37:0/1555985099 2024-04-23T18:20:28.638 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-23T18:20:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-23T18:20:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-23T18:20:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:71 bytes copied, 2.449e-05 s, 2.9 MB/s 2024-04-23T18:20:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:20:28.646 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:20:28.656 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:20:28.656 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:20:28.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/3395123919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 msgr2=0x7f0d881024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/3395123919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d881024c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009900 tx=0x7f0d8402a410 comp rx=0 tx=0).stop 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/3395123919 shutdown_connections 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/3395123919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 0x7f0d88109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/3395123919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d881024c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/3395123919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 0x7f0d88101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.766+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/3395123919 >> 172.21.15.37:0/3395123919 conn(0x7f0d880fc420 msgr2=0x7f0d880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.767+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/3395123919 shutdown_connections 2024-04-23T18:20:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.767+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/3395123919 wait complete. 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.767+0000 7f0d8ffe2640 1 Processor -- start 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.767+0000 7f0d8ffe2640 1 -- start start 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8ffe2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 0x7f0d88197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8ffe2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d88197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8ffe2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 0x7f0d8819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8ffe2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8810c9f0 con 0x7f0d88102040 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8ffe2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8810c870 con 0x7f0d88100e60 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8ffe2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8810cb70 con 0x7f0d88102a00 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d88197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8dd7e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 0x7f0d88197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d88197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36386/0 (socket says 172.21.15.37:36386) 2024-04-23T18:20:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 -- 172.21.15.37:0/803113920 learned_addr learned my addr 172.21.15.37:0/803113920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8e57f640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 0x7f0d8819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 -- 172.21.15.37:0/803113920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 msgr2=0x7f0d8819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 0x7f0d8819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 -- 172.21.15.37:0/803113920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 msgr2=0x7f0d88197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 0x7f0d88197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 -- 172.21.15.37:0/803113920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8819ec60 con 0x7f0d88102040 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8dd7e640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 0x7f0d88197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8e57f640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 0x7f0d8819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d8d57d640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d88197580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005940 tx=0x7f0d84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84046070 con 0x7f0d88102040 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.768+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84003aa0 con 0x7f0d88102040 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.769+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d88069600 con 0x7f0d88102040 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.769+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84035050 con 0x7f0d88102040 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.769+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d88069b40 con 0x7f0d88102040 2024-04-23T18:20:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.770+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8804fce0 con 0x7f0d88102040 2024-04-23T18:20:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.771+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0d84049070 con 0x7f0d88102040 2024-04-23T18:20:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.772+0000 7f0d7effd640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 0x7f0d740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.772+0000 7f0d8dd7e640 1 -- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 msgr2=0x7f0d740a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.772+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f0d840f2120 con 0x7f0d88102040 2024-04-23T18:20:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.772+0000 7f0d8dd7e640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 0x7f0d740a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:20:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.776+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d8407dff0 con 0x7f0d88102040 2024-04-23T18:20:28.837 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:28.523+0000 7fc05c7a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:20:28.837 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:28.656+0000 7fc05c7a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:20:28.837 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:28.751+0000 7fc05c7a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:20:28.855 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:28.523+0000 7fdeeb065200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:20:28.855 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:28.657+0000 7fdeeb065200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:20:28.855 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:28.753+0000 7fdeeb065200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:20:28.870 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:28 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:28.624+0000 7fbbb9e06200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:20:28.870 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:28 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:28.726+0000 7fbbb9e06200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:20:28.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.973+0000 7f0d8dd7e640 1 -- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 msgr2=0x7f0d740a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:28.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:28.973+0000 7f0d8dd7e640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 0x7f0d740a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:20:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.008+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f0d8819eef0 con 0x7f0d88102040 2024-04-23T18:20:29.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.060+0000 7f0d7effd640 1 -- 172.21.15.37:0/803113920 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) ==== 117+0+0 (secure 0 0 0) 0x7f0d840be440 con 0x7f0d88102040 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 msgr2=0x7f0d740a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 0x7f0d740a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 msgr2=0x7f0d88197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d88197580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005940 tx=0x7f0d84004270 comp rx=0 tx=0).stop 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 shutdown_connections 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f0d740a3d20 0x7f0d740a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d88102a00 0x7f0d8819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88102040 0x7f0d88197580 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 --2- 172.21.15.37:0/803113920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88100e60 0x7f0d88197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 >> 172.21.15.37:0/803113920 conn(0x7f0d880fc420 msgr2=0x7f0d881053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 shutdown_connections 2024-04-23T18:20:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.062+0000 7f0d8ffe2640 1 -- 172.21.15.37:0/803113920 wait complete. 2024-04-23T18:20:29.068 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-23T18:20:29.079 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:20:29.088 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:20:29.088 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:20:29.088 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-23T18:20:29.166 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:29 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/803113920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-23T18:20:29.166 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:28.868+0000 7fc05c7a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:20:29.170 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:29 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/803113920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-23T18:20:29.171 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:28.870+0000 7fdeeb065200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 -- 172.21.15.37:0/3233878073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8100260 msgr2=0x7f7db8100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 --2- 172.21.15.37:0/3233878073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8100260 0x7f7db8100680 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009cd0 tx=0x7f7da802a8e0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 -- 172.21.15.37:0/3233878073 shutdown_connections 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 --2- 172.21.15.37:0/3233878073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7db8103bf0 0x7f7db810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 --2- 172.21.15.37:0/3233878073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8103230 0x7f7db81036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 --2- 172.21.15.37:0/3233878073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8100260 0x7f7db8100680 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 -- 172.21.15.37:0/3233878073 >> 172.21.15.37:0/3233878073 conn(0x7f7db80fbbb0 msgr2=0x7f7db80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 -- 172.21.15.37:0/3233878073 shutdown_connections 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 -- 172.21.15.37:0/3233878073 wait complete. 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.200+0000 7f7dbdf76640 1 Processor -- start 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 -- start start 2024-04-23T18:20:29.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8100260 0x7f7db8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8103230 0x7f7db8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7db8103bf0 0x7f7db819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810db00 con 0x7f7db8103230 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810d980 con 0x7f7db8103bf0 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7dbdf76640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7db810dc80 con 0x7f7db8100260 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8103230 0x7f7db8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8103230 0x7f7db8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36392/0 (socket says 172.21.15.37:36392) 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db6ffd640 1 -- 172.21.15.37:0/2622838433 learned_addr learned my addr 172.21.15.37:0/2622838433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8100260 0x7f7db8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db7fff640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7db8103bf0 0x7f7db819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7db8103bf0 msgr2=0x7f7db819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7db8103bf0 0x7f7db819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8103230 msgr2=0x7f7db8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8103230 0x7f7db8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 -- 172.21.15.37:0/2622838433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db819fde0 con 0x7f7db8100260 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db6ffd640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db8103230 0x7f7db8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.201+0000 7f7db77fe640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8100260 0x7f7db8198210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000f80 tx=0x7f7da80093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.202+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8004040 con 0x7f7db8100260 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.202+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db81a0070 con 0x7f7db8100260 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.202+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da8004690 con 0x7f7db8100260 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.202+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8030e10 con 0x7f7db8100260 2024-04-23T18:20:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.202+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db81a05e0 con 0x7f7db8100260 2024-04-23T18:20:29.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.203+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8109480 con 0x7f7db8100260 2024-04-23T18:20:29.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.204+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7da80305a0 con 0x7f7db8100260 2024-04-23T18:20:29.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.204+0000 7f7db4ff9640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 0x7f7da00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:29.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.205+0000 7f7db6ffd640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 msgr2=0x7f7da00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:29.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.205+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f7da802e030 con 0x7f7db8100260 2024-04-23T18:20:29.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.205+0000 7f7db6ffd640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 0x7f7da00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:20:29.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.208+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7da80bdfc0 con 0x7f7db8100260 2024-04-23T18:20:29.355 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:29 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/803113920' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-23T18:20:29.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:28 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:28.988+0000 7fbbb9e06200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:20:29.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:29 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:29.132+0000 7fbbb9e06200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:20:29.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:29 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:29.236+0000 7fbbb9e06200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:20:29.355 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:29 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:29.361+0000 7fbbb9e06200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:20:29.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.406+0000 7f7db6ffd640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 msgr2=0x7f7da00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:29.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.406+0000 7f7db6ffd640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 0x7f7da00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:20:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.417+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 --> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7db8062390 con 0x7f7da00a3ca0 2024-04-23T18:20:29.587 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:29.172+0000 7fc05c7a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:20:29.605 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:29.177+0000 7fdeeb065200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:20:29.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.807+0000 7f7db6ffd640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 msgr2=0x7f7da00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3159991179 2024-04-23T18:20:29.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:29.807+0000 7f7db6ffd640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 0x7f7da00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:20:30.060 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:29 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:29.675+0000 7fbbb9e06200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:20:30.060 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:29.779+0000 7fdeeb065200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:20:30.060 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:20:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z[34769]: 2024-04-23T18:20:29.997+0000 7fdeeb065200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:20:30.060 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:29.775+0000 7fc05c7a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:20:30.061 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:20:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y[33834]: 2024-04-23T18:20:29.990+0000 7fc05c7a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:20:30.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:30.079+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f7da80b9f30 con 0x7f7db8100260 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/803113920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: Standby manager daemon y restarted 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: Standby manager daemon y started 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:20:30.313 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: Standby manager daemon z restarted 2024-04-23T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: Standby manager daemon z started 2024-04-23T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-mon[29404]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:20:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/803113920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-23T18:20:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:20:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: Standby manager daemon y restarted 2024-04-23T18:20:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: Standby manager daemon y started 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: Standby manager daemon z restarted 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: Standby manager daemon z started 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:20:30.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:30 smithi172 ceph-mon[32456]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/803113920' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: Standby manager daemon y restarted 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: Standby manager daemon y started 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:20:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.172:0/4131543967' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: Standby manager daemon z restarted 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: Standby manager daemon z started 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:20:30.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:30 smithi173 ceph-mon[32906]: from='mgr.? 172.21.15.173:0/3310512345' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:20:30.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:30.567+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f7da80ef130 con 0x7f7db8100260 2024-04-23T18:20:30.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:30.567+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 msgr2=0x7f7da00a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:20:30.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:30.567+0000 7f7db4ff9640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/3159991179,v1:172.21.15.37:6801/3159991179] conn(0x7f7da00a3ca0 0x7f7da00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:30.620 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.318+0000 7fbbb9e06200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:20:30.620 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.549+0000 7fbbb9e06200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.798+0000 7fbb1c740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.798+0000 7fbb1c740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.798+0000 7fbb1c740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.798+0000 7fbb1c740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.798+0000 7fbb1c740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.799+0000 7fbb18f39700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.799+0000 7fbb18f39700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.799+0000 7fbb18f39700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.799+0000 7fbb18f39700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.067 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:20:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:20:30.799+0000 7fbb18f39700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: Active manager daemon x restarted 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: Activating manager daemon x 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: mgrmap e21: x(active, starting, since 0.0115838s), standbys: z, y 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:20:31.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:20:31.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: Manager daemon x is now available 2024-04-23T18:20:31.340 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:31.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:20:31.341 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:20:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: Active manager daemon x restarted 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: Activating manager daemon x 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: mgrmap e21: x(active, starting, since 0.0115838s), standbys: z, y 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:20:31.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:20:31.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:20:31.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:20:31.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:20:31.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:20:31.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: Manager daemon x is now available 2024-04-23T18:20:31.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:31.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:20:31.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: Active manager daemon x restarted 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: Activating manager daemon x 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: mgrmap e21: x(active, starting, since 0.0115838s), standbys: z, y 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:20:31.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:20:31.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: Manager daemon x is now available 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:20:31.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:20:31.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.568+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mon.1 v2:172.21.15.173:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7da80c60a0 con 0x7f7db8100260 2024-04-23T18:20:31.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.569+0000 7f7db4ff9640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7da0034b20 0x7f7da0034f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:31.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.569+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7da802cec0 con 0x7f7da0034b20 2024-04-23T18:20:31.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.573+0000 7f7db6ffd640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7da0034b20 0x7f7da0034f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:31.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.575+0000 7f7db6ffd640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7da0034b20 0x7f7da0034f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7db81996d0 tx=0x7f7dac005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.644+0000 7f7db4ff9640 1 -- 172.21.15.37:0/2622838433 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7da802cec0 con 0x7f7da0034b20 2024-04-23T18:20:31.638 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-04-23T18:20:31.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7da0034b20 msgr2=0x7f7da0034f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:31.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7da0034b20 0x7f7da0034f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7db81996d0 tx=0x7f7dac005f90 comp rx=0 tx=0).stop 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8100260 msgr2=0x7f7db8198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 --2- 172.21.15.37:0/2622838433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8100260 0x7f7db8198210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000f80 tx=0x7f7da80093f0 comp rx=0 tx=0).stop 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7db7fff640 1 -- 172.21.15.37:0/2622838433 reap_dead start 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 shutdown_connections 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 >> 172.21.15.37:0/2622838433 conn(0x7f7db80fbbb0 msgr2=0x7f7db8106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 shutdown_connections 2024-04-23T18:20:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.646+0000 7f7dbdf76640 1 -- 172.21.15.37:0/2622838433 wait complete. 2024-04-23T18:20:31.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-23T18:20:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.762+0000 7fbc93efd640 1 -- 172.21.15.37:0/922760382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101180 msgr2=0x7fbc8c1015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.762+0000 7fbc93efd640 1 --2- 172.21.15.37:0/922760382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101180 0x7fbc8c1015e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009900 tx=0x7fbc7c02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- 172.21.15.37:0/922760382 shutdown_connections 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 --2- 172.21.15.37:0/922760382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc8c101b20 0x7fbc8c101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 --2- 172.21.15.37:0/922760382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101180 0x7fbc8c1015e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 --2- 172.21.15.37:0/922760382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc8c100820 0x7fbc8c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- 172.21.15.37:0/922760382 >> 172.21.15.37:0/922760382 conn(0x7fbc8c0fc420 msgr2=0x7fbc8c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- 172.21.15.37:0/922760382 shutdown_connections 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- 172.21.15.37:0/922760382 wait complete. 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 Processor -- start 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- start start 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc8c100820 0x7fbc8c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc8c101180 0x7fbc8c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 0x7fbc8c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10fcd0 con 0x7fbc8c101b20 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10fb50 con 0x7fbc8c100820 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.763+0000 7fbc93efd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10fe50 con 0x7fbc8c101180 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 0x7fbc8c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 0x7fbc8c1a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36558/0 (socket says 172.21.15.37:36558) 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 -- 172.21.15.37:0/2611962908 learned_addr learned my addr 172.21.15.37:0/2611962908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc91c99640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc8c100820 0x7fbc8c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc91498640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc8c101180 0x7fbc8c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 -- 172.21.15.37:0/2611962908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc8c101180 msgr2=0x7fbc8c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc8c101180 0x7fbc8c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 -- 172.21.15.37:0/2611962908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc8c100820 msgr2=0x7fbc8c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc8c100820 0x7fbc8c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 -- 172.21.15.37:0/2611962908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c1a1f70 con 0x7fbc8c101b20 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc9249a640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 0x7fbc8c1a1900 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e970 tx=0x7fbc8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400ccc0 con 0x7fbc8c101b20 2024-04-23T18:20:31.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8400ce60 con 0x7fbc8c101b20 2024-04-23T18:20:31.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84010430 con 0x7fbc8c101b20 2024-04-23T18:20:31.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c1a2260 con 0x7fbc8c101b20 2024-04-23T18:20:31.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.764+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a2730 con 0x7fbc8c101b20 2024-04-23T18:20:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.765+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c10b570 con 0x7fbc8c101b20 2024-04-23T18:20:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.766+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fbc84004fb0 con 0x7fbc8c101b20 2024-04-23T18:20:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.767+0000 7fbc82ffd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbc780a3a60 0x7fbc780a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.767+0000 7fbc91c99640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbc780a3a60 0x7fbc780a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.767+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fbc840d0700 con 0x7fbc8c101b20 2024-04-23T18:20:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.768+0000 7fbc91c99640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbc780a3a60 0x7fbc780a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc88005fd0 tx=0x7fbc88009680 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.771+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc84053ff0 con 0x7fbc8c101b20 2024-04-23T18:20:31.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:31.991+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fbc8c062390 con 0x7fbc780a3a60 2024-04-23T18:20:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.049+0000 7fbc82ffd640 1 -- 172.21.15.37:0/2611962908 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fbc8c062390 con 0x7fbc780a3a60 2024-04-23T18:20:32.043 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbc780a3a60 msgr2=0x7fbc780a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbc780a3a60 0x7fbc780a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc88005fd0 tx=0x7fbc88009680 comp rx=0 tx=0).stop 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 msgr2=0x7fbc8c1a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 0x7fbc8c1a1900 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e970 tx=0x7fbc8400ed70 comp rx=0 tx=0).stop 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 shutdown_connections 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbc780a3a60 0x7fbc780a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc8c101b20 0x7fbc8c1a1900 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc8c101180 0x7fbc8c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 --2- 172.21.15.37:0/2611962908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc8c100820 0x7fbc8c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 >> 172.21.15.37:0/2611962908 conn(0x7fbc8c0fc420 msgr2=0x7fbc8c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.051+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 shutdown_connections 2024-04-23T18:20:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.052+0000 7fbc93efd640 1 -- 172.21.15.37:0/2611962908 wait complete. 2024-04-23T18:20:32.056 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 -- 172.21.15.37:0/4282468655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 msgr2=0x7ff7fc107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 --2- 172.21.15.37:0/4282468655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc107770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009900 tx=0x7ff7f802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 -- 172.21.15.37:0/4282468655 shutdown_connections 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 --2- 172.21.15.37:0/4282468655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7fc107cb0 0x7ff7fc10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 --2- 172.21.15.37:0/4282468655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc107770 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 --2- 172.21.15.37:0/4282468655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 0x7ff7fc104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 -- 172.21.15.37:0/4282468655 >> 172.21.15.37:0/4282468655 conn(0x7ff7fc0fc440 msgr2=0x7ff7fc0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 -- 172.21.15.37:0/4282468655 shutdown_connections 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 -- 172.21.15.37:0/4282468655 wait complete. 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 Processor -- start 2024-04-23T18:20:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.168+0000 7ff803729640 1 -- start start 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 0x7ff7fc198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7fc107cb0 0x7ff7fc19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc10db60 con 0x7ff7fc105380 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc10d9e0 con 0x7ff7fc102a50 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc10dce0 con 0x7ff7fc107cb0 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36570/0 (socket says 172.21.15.37:36570) 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff8014c5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 0x7ff7fc198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 -- 172.21.15.37:0/3258432266 learned_addr learned my addr 172.21.15.37:0/3258432266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff801cc6640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7fc107cb0 0x7ff7fc19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 -- 172.21.15.37:0/3258432266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7fc107cb0 msgr2=0x7ff7fc19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7fc107cb0 0x7ff7fc19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 -- 172.21.15.37:0/3258432266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 msgr2=0x7ff7fc198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 0x7ff7fc198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 -- 172.21.15.37:0/3258432266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc19fd50 con 0x7ff7fc105380 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff8014c5640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 0x7ff7fc198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff800cc4640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc198670 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8031500 tx=0x7ff7f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f803f070 con 0x7ff7fc105380 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f8031c50 con 0x7ff7fc105380 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc19ffe0 con 0x7ff7fc105380 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.169+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8035030 con 0x7ff7fc105380 2024-04-23T18:20:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.170+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc1a0440 con 0x7ff7fc105380 2024-04-23T18:20:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.171+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7fc0fdf30 con 0x7ff7fc105380 2024-04-23T18:20:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.173+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ff7f8042070 con 0x7ff7fc105380 2024-04-23T18:20:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.173+0000 7ff7f27fc640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff7e80a3a60 0x7ff7e80a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.173+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff7f80f0e10 con 0x7ff7fc105380 2024-04-23T18:20:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.173+0000 7ff8014c5640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff7e80a3a60 0x7ff7e80a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.174+0000 7ff8014c5640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff7e80a3a60 0x7ff7e80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009fc0 tx=0x7ff7ec005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.177+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7f80bd380 con 0x7ff7fc105380 2024-04-23T18:20:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.405+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff7fc062390 con 0x7ff7e80a3a60 2024-04-23T18:20:32.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.413+0000 7ff7f27fc640 1 -- 172.21.15.37:0/3258432266 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7ff7fc062390 con 0x7ff7e80a3a60 2024-04-23T18:20:32.407 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:20:31", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff7e80a3a60 msgr2=0x7ff7e80a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff7e80a3a60 0x7ff7e80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009fc0 tx=0x7ff7ec005f90 comp rx=0 tx=0).stop 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 msgr2=0x7ff7fc198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc198670 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8031500 tx=0x7ff7f8003c10 comp rx=0 tx=0).stop 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 shutdown_connections 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff7e80a3a60 0x7ff7e80a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7fc107cb0 0x7ff7fc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7fc105380 0x7ff7fc198670 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 --2- 172.21.15.37:0/3258432266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7fc102a50 0x7ff7fc198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 >> 172.21.15.37:0/3258432266 conn(0x7ff7fc0fc440 msgr2=0x7ff7fc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:32.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 shutdown_connections 2024-04-23T18:20:32.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.416+0000 7ff803729640 1 -- 172.21.15.37:0/3258432266 wait complete. 2024-04-23T18:20:32.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.544+0000 7f2d234cf640 1 -- 172.21.15.37:0/4212791513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102a00 msgr2=0x7f2d1c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.544+0000 7f2d234cf640 1 --2- 172.21.15.37:0/4212791513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102a00 0x7f2d1c109320 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009ca0 tx=0x7f2d1402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.544+0000 7f2d234cf640 1 -- 172.21.15.37:0/4212791513 shutdown_connections 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.544+0000 7f2d234cf640 1 --2- 172.21.15.37:0/4212791513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102a00 0x7f2d1c109320 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.544+0000 7f2d234cf640 1 --2- 172.21.15.37:0/4212791513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d1c102040 0x7f2d1c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.544+0000 7f2d234cf640 1 --2- 172.21.15.37:0/4212791513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1c100e60 0x7f2d1c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.545+0000 7f2d234cf640 1 -- 172.21.15.37:0/4212791513 >> 172.21.15.37:0/4212791513 conn(0x7f2d1c0fc420 msgr2=0x7f2d1c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.545+0000 7f2d234cf640 1 -- 172.21.15.37:0/4212791513 shutdown_connections 2024-04-23T18:20:32.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.545+0000 7f2d234cf640 1 -- 172.21.15.37:0/4212791513 wait complete. 2024-04-23T18:20:32.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.545+0000 7f2d234cf640 1 Processor -- start 2024-04-23T18:20:32.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 -- start start 2024-04-23T18:20:32.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d1c100e60 0x7f2d1c06a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 0x7f2d1c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1c102a00 0x7f2d1c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c10ca50 con 0x7f2d1c102040 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c10c8d0 con 0x7f2d1c102a00 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d234cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c10cbd0 con 0x7f2d1c100e60 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 0x7f2d1c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 0x7f2d1c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36598/0 (socket says 172.21.15.37:36598) 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 -- 172.21.15.37:0/497420042 learned_addr learned my addr 172.21.15.37:0/497420042 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d2126b640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d1c100e60 0x7f2d1c06a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d21a6c640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1c102a00 0x7f2d1c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 -- 172.21.15.37:0/497420042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d1c100e60 msgr2=0x7f2d1c06a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d1c100e60 0x7f2d1c06a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 -- 172.21.15.37:0/497420042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1c102a00 msgr2=0x7f2d1c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1c102a00 0x7f2d1c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 -- 172.21.15.37:0/497420042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c19fd60 con 0x7f2d1c102040 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.546+0000 7f2d20a6a640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 0x7f2d1c19ce60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9f0 tx=0x7f2d0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.547+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00cd80 con 0x7f2d1c102040 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.547+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c0054d0 con 0x7f2d1c102040 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.547+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c19fff0 con 0x7f2d1c102040 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.547+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c010690 con 0x7f2d1c102040 2024-04-23T18:20:32.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.547+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1a04c0 con 0x7f2d1c102040 2024-04-23T18:20:32.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.548+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c108200 con 0x7f2d1c102040 2024-04-23T18:20:32.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.549+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f2d0c004e90 con 0x7f2d1c102040 2024-04-23T18:20:32.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.550+0000 7f2d127fc640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2d080a3a60 0x7f2d080a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.550+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f2d0c0d0460 con 0x7f2d1c102040 2024-04-23T18:20:32.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.550+0000 7f2d2126b640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2d080a3a60 0x7f2d080a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.550+0000 7f2d2126b640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2d080a3a60 0x7f2d080a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005fd0 tx=0x7f2d1800b410 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:32.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.555+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d0c09d020 con 0x7f2d1c102040 2024-04-23T18:20:32.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.578+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f2d0c098f90 con 0x7f2d1c102040 2024-04-23T18:20:32.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.791+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2d1c062390 con 0x7f2d080a3a60 2024-04-23T18:20:32.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.798+0000 7f2d127fc640 1 -- 172.21.15.37:0/497420042 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2d1c062390 con 0x7f2d080a3a60 2024-04-23T18:20:32.792 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.800+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2d080a3a60 msgr2=0x7f2d080a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.800+0000 7f2d234cf640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2d080a3a60 0x7f2d080a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005fd0 tx=0x7f2d1800b410 comp rx=0 tx=0).stop 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.800+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 msgr2=0x7f2d1c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.800+0000 7f2d234cf640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 0x7f2d1c19ce60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9f0 tx=0x7f2d0c00edf0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 shutdown_connections 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2d080a3a60 0x7f2d080a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1c102a00 0x7f2d1c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c102040 0x7f2d1c19ce60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 --2- 172.21.15.37:0/497420042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d1c100e60 0x7f2d1c06a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 >> 172.21.15.37:0/497420042 conn(0x7f2d1c0fc420 msgr2=0x7f2d1c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 shutdown_connections 2024-04-23T18:20:32.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.801+0000 7f2d234cf640 1 -- 172.21.15.37:0/497420042 wait complete. 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: [23/Apr/2024:18:20:31] ENGINE Bus STARTING 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: [23/Apr/2024:18:20:31] ENGINE Serving on http://172.21.15.37:8765 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: [23/Apr/2024:18:20:31] ENGINE Serving on https://172.21.15.37:7150 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: [23/Apr/2024:18:20:31] ENGINE Bus STARTED 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: mgrmap e22: x(active, since 1.02259s), standbys: z, y 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:32.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:32 smithi037 ceph-mon[29404]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:32.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-23T18:20:32.832 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-23T18:20:32.846 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-23T18:20:32.856 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:20:32.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-23T18:20:32.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: [23/Apr/2024:18:20:31] ENGINE Bus STARTING 2024-04-23T18:20:32.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: [23/Apr/2024:18:20:31] ENGINE Serving on http://172.21.15.37:8765 2024-04-23T18:20:32.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: [23/Apr/2024:18:20:31] ENGINE Serving on https://172.21.15.37:7150 2024-04-23T18:20:32.857 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: [23/Apr/2024:18:20:31] ENGINE Bus STARTED 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: mgrmap e22: x(active, since 1.02259s), standbys: z, y 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:32 smithi173 ceph-mon[32906]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: [23/Apr/2024:18:20:31] ENGINE Bus STARTING 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: [23/Apr/2024:18:20:31] ENGINE Serving on http://172.21.15.37:8765 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: [23/Apr/2024:18:20:31] ENGINE Serving on https://172.21.15.37:7150 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: [23/Apr/2024:18:20:31] ENGINE Bus STARTED 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: mgrmap e22: x(active, since 1.02259s), standbys: z, y 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:32.858 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:32 smithi172 ceph-mon[32456]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:32.874 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-23T18:20:32.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:32.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 -- 172.21.15.37:0/852217692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c01029a0 msgr2=0x7f87c01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 --2- 172.21.15.37:0/852217692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c01029a0 0x7f87c01092c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009ca0 tx=0x7f87b802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 -- 172.21.15.37:0/852217692 shutdown_connections 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 --2- 172.21.15.37:0/852217692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c01029a0 0x7f87c01092c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 --2- 172.21.15.37:0/852217692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c0101fe0 0x7f87c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 --2- 172.21.15.37:0/852217692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87c0100e90 0x7f87c01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 -- 172.21.15.37:0/852217692 >> 172.21.15.37:0/852217692 conn(0x7f87c00fc400 msgr2=0x7f87c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 -- 172.21.15.37:0/852217692 shutdown_connections 2024-04-23T18:20:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.997+0000 7f87c4a35640 1 -- 172.21.15.37:0/852217692 wait complete. 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 Processor -- start 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 -- start start 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 0x7f87c019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c0101fe0 0x7f87c019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87c01029a0 0x7f87c01a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87c010c9f0 con 0x7f87c0100e90 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87c010c870 con 0x7f87c0101fe0 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87c4a35640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f87c010cb70 con 0x7f87c01029a0 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 0x7f87c019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 0x7f87c019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36626/0 (socket says 172.21.15.37:36626) 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87be59c640 1 -- 172.21.15.37:0/2502877341 learned_addr learned my addr 172.21.15.37:0/2502877341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87bed9d640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87c01029a0 0x7f87c01a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.998+0000 7f87bdd9b640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c0101fe0 0x7f87c019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87be59c640 1 -- 172.21.15.37:0/2502877341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87c01029a0 msgr2=0x7f87c01a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87be59c640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87c01029a0 0x7f87c01a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87be59c640 1 -- 172.21.15.37:0/2502877341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c0101fe0 msgr2=0x7f87c019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87be59c640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c0101fe0 0x7f87c019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87be59c640 1 -- 172.21.15.37:0/2502877341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c01a2040 con 0x7f87c0100e90 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87be59c640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 0x7f87c019a420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a800cd80 con 0x7f87c0100e90 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a80054d0 con 0x7f87c0100e90 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c01a2330 con 0x7f87c0100e90 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8010690 con 0x7f87c0100e90 2024-04-23T18:20:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:32.999+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c01a2830 con 0x7f87c0100e90 2024-04-23T18:20:32.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.000+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c01a2af0 con 0x7f87c0100e90 2024-04-23T18:20:32.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.001+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f87a8004e90 con 0x7f87c0100e90 2024-04-23T18:20:32.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.002+0000 7f87b77fe640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f87ac0a3d20 0x7f87ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:32.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.002+0000 7f87bdd9b640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f87ac0a3d20 0x7f87ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:32.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.002+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f87a80d0c40 con 0x7f87c0100e90 2024-04-23T18:20:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.003+0000 7f87bdd9b640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f87ac0a3d20 0x7f87ac0a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f87c019b930 tx=0x7f87b0005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.006+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87a809d290 con 0x7f87c0100e90 2024-04-23T18:20:33.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.218+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87c01a2cc0 con 0x7f87c0100e90 2024-04-23T18:20:33.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.221+0000 7f87b77fe640 1 -- 172.21.15.37:0/2502877341 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23061 (secure 0 0 0) 0x7f87a809cb90 con 0x7f87c0100e90 2024-04-23T18:20:33.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:33.215 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:20:30.552696+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f87ac0a3d20 msgr2=0x7f87ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f87ac0a3d20 0x7f87ac0a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f87c019b930 tx=0x7f87b0005f90 comp rx=0 tx=0).stop 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 msgr2=0x7f87c019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 0x7f87c019a420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e9f0 tx=0x7f87a800edf0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 shutdown_connections 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f87ac0a3d20 0x7f87ac0a61e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87c01029a0 0x7f87c01a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87c0101fe0 0x7f87c019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 --2- 172.21.15.37:0/2502877341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87c0100e90 0x7f87c019a420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.223+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 >> 172.21.15.37:0/2502877341 conn(0x7f87c00fc400 msgr2=0x7f87c00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.224+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 shutdown_connections 2024-04-23T18:20:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.224+0000 7f87c4a35640 1 -- 172.21.15.37:0/2502877341 wait complete. 2024-04-23T18:20:33.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:20:33.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 -- 172.21.15.37:0/1719167569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101fe0 msgr2=0x7f35fc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 --2- 172.21.15.37:0/1719167569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101fe0 0x7f35fc102460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009900 tx=0x7f35f002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 -- 172.21.15.37:0/1719167569 shutdown_connections 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 --2- 172.21.15.37:0/1719167569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35fc1029a0 0x7f35fc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 --2- 172.21.15.37:0/1719167569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101fe0 0x7f35fc102460 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 --2- 172.21.15.37:0/1719167569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35fc100e90 0x7f35fc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 -- 172.21.15.37:0/1719167569 >> 172.21.15.37:0/1719167569 conn(0x7f35fc0fc400 msgr2=0x7f35fc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 -- 172.21.15.37:0/1719167569 shutdown_connections 2024-04-23T18:20:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.349+0000 7f3600936640 1 -- 172.21.15.37:0/1719167569 wait complete. 2024-04-23T18:20:33.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 Processor -- start 2024-04-23T18:20:33.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 -- start start 2024-04-23T18:20:33.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 0x7f35fc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35fc101fe0 0x7f35fc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35fc1029a0 0x7f35fc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10c9f0 con 0x7f35fc100e90 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10c870 con 0x7f35fc101fe0 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.350+0000 7f3600936640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10cb70 con 0x7f35fc1029a0 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 0x7f35fc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 0x7f35fc198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36652/0 (socket says 172.21.15.37:36652) 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 -- 172.21.15.37:0/2321535316 learned_addr learned my addr 172.21.15.37:0/2321535316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35f9d9b640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35fc101fe0 0x7f35fc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fad9d640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35fc1029a0 0x7f35fc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 -- 172.21.15.37:0/2321535316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35fc1029a0 msgr2=0x7f35fc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35fc1029a0 0x7f35fc19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 -- 172.21.15.37:0/2321535316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35fc101fe0 msgr2=0x7f35fc1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35fc101fe0 0x7f35fc1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 -- 172.21.15.37:0/2321535316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc19fe90 con 0x7f35fc100e90 2024-04-23T18:20:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35fa59c640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 0x7f35fc198270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9f0 tx=0x7f35ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00cd80 con 0x7f35fc100e90 2024-04-23T18:20:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ec0054d0 con 0x7f35fc100e90 2024-04-23T18:20:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec010690 con 0x7f35fc100e90 2024-04-23T18:20:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc1a0180 con 0x7f35fc100e90 2024-04-23T18:20:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.351+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1a06a0 con 0x7f35fc100e90 2024-04-23T18:20:33.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.352+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35fc1a0960 con 0x7f35fc100e90 2024-04-23T18:20:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.354+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f35ec004e30 con 0x7f35fc100e90 2024-04-23T18:20:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.354+0000 7f35eb7fe640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f35e40a3da0 0x7f35e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.354+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f35ec0d0d80 con 0x7f35fc100e90 2024-04-23T18:20:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.354+0000 7f35f9d9b640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f35e40a3da0 0x7f35e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.355+0000 7f35f9d9b640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f35e40a3da0 0x7f35e40a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009230 tx=0x7f35f00314a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:33.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.358+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35ec053ff0 con 0x7f35fc100e90 2024-04-23T18:20:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.547+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35fc1a0b30 con 0x7f35fc100e90 2024-04-23T18:20:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.547+0000 7f35eb7fe640 1 -- 172.21.15.37:0/2321535316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f35ec09d430 con 0x7f35fc100e90 2024-04-23T18:20:33.541 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:20:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f35e40a3da0 msgr2=0x7f35e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f35e40a3da0 0x7f35e40a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009230 tx=0x7f35f00314a0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 msgr2=0x7f35fc198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 0x7f35fc198270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00e9f0 tx=0x7f35ec00edf0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 shutdown_connections 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f35e40a3da0 0x7f35e40a6260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35fc1029a0 0x7f35fc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35fc101fe0 0x7f35fc1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 --2- 172.21.15.37:0/2321535316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc100e90 0x7f35fc198270 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 >> 172.21.15.37:0/2321535316 conn(0x7f35fc0fc400 msgr2=0x7f35fc06a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 shutdown_connections 2024-04-23T18:20:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.549+0000 7f3600936640 1 -- 172.21.15.37:0/2321535316 wait complete. 2024-04-23T18:20:33.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:20:33.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 -- 172.21.15.37:0/2917739148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 msgr2=0x7fd3ac10a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 --2- 172.21.15.37:0/2917739148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac10a0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009ca0 tx=0x7fd3a402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 -- 172.21.15.37:0/2917739148 shutdown_connections 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 --2- 172.21.15.37:0/2917739148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac10a0a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 --2- 172.21.15.37:0/2917739148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac105320 0x7fd3ac107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 --2- 172.21.15.37:0/2917739148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac1029f0 0x7fd3ac104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 -- 172.21.15.37:0/2917739148 >> 172.21.15.37:0/2917739148 conn(0x7fd3ac0fc3e0 msgr2=0x7fd3ac0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 -- 172.21.15.37:0/2917739148 shutdown_connections 2024-04-23T18:20:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.669+0000 7fd3b1130640 1 -- 172.21.15.37:0/2917739148 wait complete. 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 Processor -- start 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 -- start start 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac1029f0 0x7fd3ac102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac105320 0x7fd3ac1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10daa0 con 0x7fd3ac107c50 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10d920 con 0x7fd3ac105320 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3b1130640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10dc20 con 0x7fd3ac1029f0 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.670+0000 7fd3ab59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3ab59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36678/0 (socket says 172.21.15.37:36678) 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3ab59e640 1 -- 172.21.15.37:0/1814667810 learned_addr learned my addr 172.21.15.37:0/1814667810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac1029f0 0x7fd3ac102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aa59c640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac105320 0x7fd3ac1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 -- 172.21.15.37:0/1814667810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac105320 msgr2=0x7fd3ac1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac105320 0x7fd3ac1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 -- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 msgr2=0x7fd3ac100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 -- 172.21.15.37:0/1814667810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac101770 con 0x7fd3ac1029f0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3ab59e640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3aad9d640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac1029f0 0x7fd3ac102360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00e9f0 tx=0x7fd39c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00ccc0 con 0x7fd3ac1029f0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd39c0108e0 con 0x7fd3ac1029f0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac1a2290 con 0x7fd3ac1029f0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c018880 con 0x7fd3ac1029f0 2024-04-23T18:20:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.671+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac1a27e0 con 0x7fd3ac1029f0 2024-04-23T18:20:33.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.672+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac0fded0 con 0x7fd3ac1029f0 2024-04-23T18:20:33.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.673+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fd39c010430 con 0x7fd3ac1029f0 2024-04-23T18:20:33.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.674+0000 7fd393fff640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3940a3d20 0x7fd3940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:33.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.674+0000 7fd3aa59c640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3940a3d20 0x7fd3940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:33.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.674+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fd39c0d0bc0 con 0x7fd3ac1029f0 2024-04-23T18:20:33.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.674+0000 7fd3aa59c640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3940a3d20 0x7fd3940a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac1020d0 tx=0x7fd3a0005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.677+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd39c09cc20 con 0x7fd3ac1029f0 2024-04-23T18:20:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T18:20:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.837 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2502877341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:33.838 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:33 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2321535316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.855 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:33.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2502877341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:33.856 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:33 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2321535316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:33.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:33.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:33.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2502877341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:33.871 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:33 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2321535316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.880+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd3ac062b50 con 0x7fd3ac1029f0 2024-04-23T18:20:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.880+0000 7fd393fff640 1 -- 172.21.15.37:0/1814667810 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fd39c0a0f20 con 0x7fd3ac1029f0 2024-04-23T18:20:33.874 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:20:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3940a3d20 msgr2=0x7fd3940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3940a3d20 0x7fd3940a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac1020d0 tx=0x7fd3a0005f90 comp rx=0 tx=0).stop 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac1029f0 msgr2=0x7fd3ac102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac1029f0 0x7fd3ac102360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00e9f0 tx=0x7fd39c00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 shutdown_connections 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3940a3d20 0x7fd3940a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3ac107c50 0x7fd3ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac105320 0x7fd3ac1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 --2- 172.21.15.37:0/1814667810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3ac1029f0 0x7fd3ac102360 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 >> 172.21.15.37:0/1814667810 conn(0x7fd3ac0fc3e0 msgr2=0x7fd3ac103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 shutdown_connections 2024-04-23T18:20:33.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:33.883+0000 7fd3b1130640 1 -- 172.21.15.37:0/1814667810 wait complete. 2024-04-23T18:20:33.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:20:33.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 -- 172.21.15.37:0/196115316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4102a10 msgr2=0x7f6cf4104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/196115316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4102a10 0x7f6cf4104e00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009900 tx=0x7f6cf002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 -- 172.21.15.37:0/196115316 shutdown_connections 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/196115316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf4107c70 0x7f6cf410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/196115316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4105340 0x7f6cf4107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/196115316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4102a10 0x7f6cf4104e00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.003+0000 7f6cfbacf640 1 -- 172.21.15.37:0/196115316 >> 172.21.15.37:0/196115316 conn(0x7f6cf40fc400 msgr2=0x7f6cf40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 -- 172.21.15.37:0/196115316 shutdown_connections 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 -- 172.21.15.37:0/196115316 wait complete. 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 Processor -- start 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 -- start start 2024-04-23T18:20:34.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4102a10 0x7f6cf4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 0x7f6cf4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf4107c70 0x7f6cf419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf410dad0 con 0x7f6cf4105340 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf410d950 con 0x7f6cf4107c70 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfbacf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf410dc50 con 0x7f6cf4102a10 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cf906a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 0x7f6cf4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cf906a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 0x7f6cf4198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36696/0 (socket says 172.21.15.37:36696) 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cf906a640 1 -- 172.21.15.37:0/3769980872 learned_addr learned my addr 172.21.15.37:0/3769980872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cf986b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4102a10 0x7f6cf4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.004+0000 7f6cfa06c640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf4107c70 0x7f6cf419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cf906a640 1 -- 172.21.15.37:0/3769980872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4102a10 msgr2=0x7f6cf4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cf906a640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4102a10 0x7f6cf4198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cf906a640 1 -- 172.21.15.37:0/3769980872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf4107c70 msgr2=0x7f6cf419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cf906a640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf4107c70 0x7f6cf419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cf906a640 1 -- 172.21.15.37:0/3769980872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf419fe70 con 0x7f6cf4105340 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cf906a640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 0x7f6cf4198790 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400e9f0 tx=0x7f6ce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400ccc0 con 0x7f6cf4105340 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce40054d0 con 0x7f6cf4105340 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf41a0160 con 0x7f6cf4105340 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4010430 con 0x7f6cf4105340 2024-04-23T18:20:34.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.005+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf41a06b0 con 0x7f6cf4105340 2024-04-23T18:20:34.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.006+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf40fdef0 con 0x7f6cf4105340 2024-04-23T18:20:34.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.007+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f6ce40046c0 con 0x7f6cf4105340 2024-04-23T18:20:34.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.008+0000 7f6ceaffd640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6ce00a3da0 0x7f6ce00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.008+0000 7f6cf986b640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6ce00a3da0 0x7f6ce00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.008+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f6ce40d0b50 con 0x7f6cf4105340 2024-04-23T18:20:34.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.009+0000 7f6cf986b640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6ce00a3da0 0x7f6ce00a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00314a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:34.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.013+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ce409d1a0 con 0x7f6cf4105340 2024-04-23T18:20:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.248+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6cf402d810 con 0x7f6cf4105340 2024-04-23T18:20:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.248+0000 7f6ceaffd640 1 -- 172.21.15.37:0/3769980872 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f6ce409caa0 con 0x7f6cf4105340 2024-04-23T18:20:34.242 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6ce00a3da0 msgr2=0x7f6ce00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6ce00a3da0 0x7f6ce00a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00314a0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 msgr2=0x7f6cf4198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 0x7f6cf4198790 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400e9f0 tx=0x7f6ce400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 shutdown_connections 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6ce00a3da0 0x7f6ce00a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf4107c70 0x7f6cf419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf4105340 0x7f6cf4198790 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 --2- 172.21.15.37:0/3769980872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4102a10 0x7f6cf4198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.250+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 >> 172.21.15.37:0/3769980872 conn(0x7f6cf40fc400 msgr2=0x7f6cf41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.251+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 shutdown_connections 2024-04-23T18:20:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.251+0000 7f6cfbacf640 1 -- 172.21.15.37:0/3769980872 wait complete. 2024-04-23T18:20:34.256 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 -- 172.21.15.37:0/2253187860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 msgr2=0x7f6f1c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 --2- 172.21.15.37:0/2253187860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c1012c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009900 tx=0x7f6f0c02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 -- 172.21.15.37:0/2253187860 shutdown_connections 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 --2- 172.21.15.37:0/2253187860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c1029b0 0x7f6f1c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 --2- 172.21.15.37:0/2253187860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f1c101ff0 0x7f6f1c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 --2- 172.21.15.37:0/2253187860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c1012c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.367+0000 7f6f218cb640 1 -- 172.21.15.37:0/2253187860 >> 172.21.15.37:0/2253187860 conn(0x7f6f1c0fc430 msgr2=0x7f6f1c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.368+0000 7f6f218cb640 1 -- 172.21.15.37:0/2253187860 shutdown_connections 2024-04-23T18:20:34.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.368+0000 7f6f218cb640 1 -- 172.21.15.37:0/2253187860 wait complete. 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.368+0000 7f6f218cb640 1 Processor -- start 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 -- start start 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c101ff0 0x7f6f1c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f1c1029b0 0x7f6f1c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c9d0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c850 con 0x7f6f1c101ff0 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f218cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10cb50 con 0x7f6f1c1029b0 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c06b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36720/0 (socket says 172.21.15.37:36720) 2024-04-23T18:20:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 -- 172.21.15.37:0/1260911331 learned_addr learned my addr 172.21.15.37:0/1260911331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1a7fc640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c101ff0 0x7f6f1c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1b7fe640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f1c1029b0 0x7f6f1c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 -- 172.21.15.37:0/1260911331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f1c1029b0 msgr2=0x7f6f1c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f1c1029b0 0x7f6f1c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 -- 172.21.15.37:0/1260911331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c101ff0 msgr2=0x7f6f1c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c101ff0 0x7f6f1c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.369+0000 7f6f1affd640 1 -- 172.21.15.37:0/1260911331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c19fe50 con 0x7f6f1c100ea0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.370+0000 7f6f1affd640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c06b0e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c005940 tx=0x7f6f0c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.370+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c03f030 con 0x7f6f1c100ea0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.370+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c003aa0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.370+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1a00e0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.370+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c035070 con 0x7f6f1c100ea0 2024-04-23T18:20:34.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.370+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a05d0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.371+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c0691c0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.372+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f6f0c042030 con 0x7f6f1c100ea0 2024-04-23T18:20:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.373+0000 7f6f208c9640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6f040a3d20 0x7f6f040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.373+0000 7f6f1a7fc640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6f040a3d20 0x7f6f040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.373+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f6f0c0f0de0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.373+0000 7f6f1a7fc640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6f040a3d20 0x7f6f040a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c06ae50 tx=0x7f6f10005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.377+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f0c0bcde0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.649+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f1c062b50 con 0x7f6f1c100ea0 2024-04-23T18:20:34.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.651+0000 7f6f208c9640 1 -- 172.21.15.37:0/1260911331 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23061 (secure 0 0 0) 0x7f6f0c0c10e0 con 0x7f6f1c100ea0 2024-04-23T18:20:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:34.645 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:20:30.552696+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6f040a3d20 msgr2=0x7f6f040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6f040a3d20 0x7f6f040a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c06ae50 tx=0x7f6f10005f90 comp rx=0 tx=0).stop 2024-04-23T18:20:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 msgr2=0x7f6f1c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c06b0e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c005940 tx=0x7f6f0c031a50 comp rx=0 tx=0).stop 2024-04-23T18:20:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 shutdown_connections 2024-04-23T18:20:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6f040a3d20 0x7f6f040a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f1c1029b0 0x7f6f1c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c101ff0 0x7f6f1c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 --2- 172.21.15.37:0/1260911331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f1c100ea0 0x7f6f1c06b0e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 >> 172.21.15.37:0/1260911331 conn(0x7f6f1c0fc430 msgr2=0x7f6f1c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 shutdown_connections 2024-04-23T18:20:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.653+0000 7f6f218cb640 1 -- 172.21.15.37:0/1260911331 wait complete. 2024-04-23T18:20:34.658 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:20:34.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.773+0000 7f361ce79640 1 -- 172.21.15.37:0/451857792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618106160 msgr2=0x7f361810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.773+0000 7f361ce79640 1 --2- 172.21.15.37:0/451857792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618106160 0x7f361810dd30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3610009ca0 tx=0x7f361002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 -- 172.21.15.37:0/451857792 shutdown_connections 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 --2- 172.21.15.37:0/451857792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618106160 0x7f361810dd30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 --2- 172.21.15.37:0/451857792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3618101a40 0x7f3618105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 --2- 172.21.15.37:0/451857792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618101090 0x7f3618101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 -- 172.21.15.37:0/451857792 >> 172.21.15.37:0/451857792 conn(0x7f3618077f50 msgr2=0x7f3618078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 -- 172.21.15.37:0/451857792 shutdown_connections 2024-04-23T18:20:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.774+0000 7f361ce79640 1 -- 172.21.15.37:0/451857792 wait complete. 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 Processor -- start 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 -- start start 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618101090 0x7f3618197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3618106160 0x7f3618197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361819bef0 0x7f361819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36181110b0 con 0x7f3618101090 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3618110f30 con 0x7f361819bef0 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361ce79640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3618111230 con 0x7f3618106160 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618101090 0x7f3618197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f3616d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361819bef0 0x7f361819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618101090 0x7f3618197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42926/0 (socket says 172.21.15.37:42926) 2024-04-23T18:20:34.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f361659c640 1 -- 172.21.15.37:0/475150211 learned_addr learned my addr 172.21.15.37:0/475150211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.775+0000 7f360dd9b640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3618106160 0x7f3618197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f3616d9d640 1 -- 172.21.15.37:0/475150211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3618106160 msgr2=0x7f3618197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f3616d9d640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3618106160 0x7f3618197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f3616d9d640 1 -- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618101090 msgr2=0x7f3618197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f3616d9d640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618101090 0x7f3618197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f3616d9d640 1 -- 172.21.15.37:0/475150211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361819c9e0 con 0x7f361819bef0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f3616d9d640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361819bef0 0x7f361819c3a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f361002ade0 tx=0x7f36100319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610003bf0 con 0x7f361819bef0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3610003d90 con 0x7f361819bef0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361819cc70 con 0x7f361819bef0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361002fbd0 con 0x7f361819bef0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.776+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36180693d0 con 0x7f361819bef0 2024-04-23T18:20:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.778+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3618068d70 con 0x7f361819bef0 2024-04-23T18:20:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.779+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f361002fd70 con 0x7f361819bef0 2024-04-23T18:20:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.779+0000 7f360ffff640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f36040a3d20 0x7f36040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.779+0000 7f361659c640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f36040a3d20 0x7f36040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.779+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f36100f1c00 con 0x7f361819bef0 2024-04-23T18:20:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.780+0000 7f361659c640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f36040a3d20 0x7f36040a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3608005fd0 tx=0x7f360800b480 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:34.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.784+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36100be360 con 0x7f361819bef0 2024-04-23T18:20:34.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.842+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f36100ba2d0 con 0x7f361819bef0 2024-04-23T18:20:34.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.983+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3618101aa0 con 0x7f361819bef0 2024-04-23T18:20:34.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.984+0000 7f360ffff640 1 -- 172.21.15.37:0/475150211 <== mon.2 v2:172.21.15.172:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f36100dce20 con 0x7f361819bef0 2024-04-23T18:20:34.978 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:20:34.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f36040a3d20 msgr2=0x7f36040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f36040a3d20 0x7f36040a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3608005fd0 tx=0x7f360800b480 comp rx=0 tx=0).stop 2024-04-23T18:20:34.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361819bef0 msgr2=0x7f361819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:34.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361819bef0 0x7f361819c3a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f361002ade0 tx=0x7f36100319f0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 shutdown_connections 2024-04-23T18:20:34.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f36040a3d20 0x7f36040a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361819bef0 0x7f361819c3a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3618106160 0x7f3618197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 --2- 172.21.15.37:0/475150211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3618101090 0x7f3618197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:34.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 >> 172.21.15.37:0/475150211 conn(0x7f3618077f50 msgr2=0x7f36180fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:34.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 shutdown_connections 2024-04-23T18:20:34.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:34.986+0000 7f361ce79640 1 -- 172.21.15.37:0/475150211 wait complete. 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-23T18:20:34.991 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:34.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Unable to set osd_memory_target on smithi037 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1814667810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3769980872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:34.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:34 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1260911331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:34.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:20:35.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T18:20:35.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Unable to set osd_memory_target on smithi037 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1814667810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:35.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:35.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:35.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3769980872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.090 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:34 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1260911331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:35.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Unable to set osd_memory_target on smithi037 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1814667810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3769980872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:34 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1260911331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 -- 172.21.15.37:0/141691557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe4101090 msgr2=0x7fefe4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/141691557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe4101090 0x7fefe4101470 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009900 tx=0x7fefe002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 -- 172.21.15.37:0/141691557 shutdown_connections 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/141691557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe4106160 0x7fefe410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/141691557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe4101a40 0x7fefe4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/141691557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe4101090 0x7fefe4101470 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 -- 172.21.15.37:0/141691557 >> 172.21.15.37:0/141691557 conn(0x7fefe40771d0 msgr2=0x7fefe40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 -- 172.21.15.37:0/141691557 shutdown_connections 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.112+0000 7fefeb5bf640 1 -- 172.21.15.37:0/141691557 wait complete. 2024-04-23T18:20:35.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 Processor -- start 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 -- start start 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe4101090 0x7fefe41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe4106160 0x7fefe4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 0x7fefe419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefe41110a0 con 0x7fefe419bfb0 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefe4110f20 con 0x7fefe4106160 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefeb5bf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefe4111220 con 0x7fefe4101090 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefe9b5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 0x7fefe419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefe9b5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 0x7fefe419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42936/0 (socket says 172.21.15.37:42936) 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefe9b5c640 1 -- 172.21.15.37:0/2674091914 learned_addr learned my addr 172.21.15.37:0/2674091914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefe8b5a640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe4106160 0x7fefe4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.113+0000 7fefe935b640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe4101090 0x7fefe41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefe9b5c640 1 -- 172.21.15.37:0/2674091914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe4101090 msgr2=0x7fefe41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefe9b5c640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe4101090 0x7fefe41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefe9b5c640 1 -- 172.21.15.37:0/2674091914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe4106160 msgr2=0x7fefe4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefe9b5c640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe4106160 0x7fefe4198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefe9b5c640 1 -- 172.21.15.37:0/2674091914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe419caa0 con 0x7fefe419bfb0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefe9b5c640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 0x7fefe419c460 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009fd0 tx=0x7fefdc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc017070 con 0x7fefe419bfb0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefdc003e20 con 0x7fefe419bfb0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe419cd90 con 0x7fefe419bfb0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc010460 con 0x7fefe419bfb0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.114+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41a55e0 con 0x7fefe419bfb0 2024-04-23T18:20:35.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.115+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe4102560 con 0x7fefe419bfb0 2024-04-23T18:20:35.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.116+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fefdc003970 con 0x7fefe419bfb0 2024-04-23T18:20:35.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.117+0000 7fefda7fc640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fefd00a3db0 0x7fefd00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.117+0000 7fefe935b640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fefd00a3db0 0x7fefd00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.117+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fefdc0d0600 con 0x7fefe419bfb0 2024-04-23T18:20:35.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.117+0000 7fefe935b640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fefd00a3db0 0x7fefd00a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fefe0031440 tx=0x7fefe0005b50 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:35.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.121+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefdc09c4f0 con 0x7fefe419bfb0 2024-04-23T18:20:35.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.313+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fefe4199af0 con 0x7fefe419bfb0 2024-04-23T18:20:35.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.313+0000 7fefda7fc640 1 -- 172.21.15.37:0/2674091914 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fefe4199af0 con 0x7fefe419bfb0 2024-04-23T18:20:35.307 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:20:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fefd00a3db0 msgr2=0x7fefd00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fefd00a3db0 0x7fefd00a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fefe0031440 tx=0x7fefe0005b50 comp rx=0 tx=0).stop 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 msgr2=0x7fefe419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 0x7fefe419c460 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009fd0 tx=0x7fefdc00b260 comp rx=0 tx=0).stop 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 shutdown_connections 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fefd00a3db0 0x7fefd00a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe419bfb0 0x7fefe419c460 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe4106160 0x7fefe4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 --2- 172.21.15.37:0/2674091914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe4101090 0x7fefe41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 >> 172.21.15.37:0/2674091914 conn(0x7fefe40771d0 msgr2=0x7fefe406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 shutdown_connections 2024-04-23T18:20:35.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.315+0000 7fefeb5bf640 1 -- 172.21.15.37:0/2674091914 wait complete. 2024-04-23T18:20:35.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.431+0000 7f586ccc2640 1 -- 172.21.15.37:0/572863875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868101a40 msgr2=0x7f5868105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.431+0000 7f586ccc2640 1 --2- 172.21.15.37:0/572863875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868101a40 0x7f5868105a80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5850009900 tx=0x7f585002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 -- 172.21.15.37:0/572863875 shutdown_connections 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 --2- 172.21.15.37:0/572863875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5868106160 0x7f586810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 --2- 172.21.15.37:0/572863875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868101a40 0x7f5868105a80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 --2- 172.21.15.37:0/572863875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5868101090 0x7f5868101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 -- 172.21.15.37:0/572863875 >> 172.21.15.37:0/572863875 conn(0x7f5868077f50 msgr2=0x7f5868078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 -- 172.21.15.37:0/572863875 shutdown_connections 2024-04-23T18:20:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 -- 172.21.15.37:0/572863875 wait complete. 2024-04-23T18:20:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 Processor -- start 2024-04-23T18:20:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.432+0000 7f586ccc2640 1 -- start start 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5868101090 0x7f5868102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5868106160 0x7f5868103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 0x7f58681a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111080 con 0x7f58681035a0 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5868110f00 con 0x7f5868106160 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111200 con 0x7f5868101090 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 0x7f58681a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 0x7f58681a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42960/0 (socket says 172.21.15.37:42960) 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 -- 172.21.15.37:0/2239443925 learned_addr learned my addr 172.21.15.37:0/2239443925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f585ffff640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5868106160 0x7f5868103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586659c640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5868101090 0x7f5868102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 -- 172.21.15.37:0/2239443925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5868101090 msgr2=0x7f5868102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5868101090 0x7f5868102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 -- 172.21.15.37:0/2239443925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5868106160 msgr2=0x7f5868103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5868106160 0x7f5868103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 -- 172.21.15.37:0/2239443925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681a6fc0 con 0x7f58681035a0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f5866d9d640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 0x7f58681a68c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f586000eaa0 tx=0x7f586000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004d20 con 0x7f58681035a0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681a72b0 con 0x7f58681035a0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5860005020 con 0x7f58681035a0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600116b0 con 0x7f58681035a0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.433+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681a7780 con 0x7f58681035a0 2024-04-23T18:20:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.434+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5868068d70 con 0x7f58681035a0 2024-04-23T18:20:35.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.436+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f58600051d0 con 0x7f58681035a0 2024-04-23T18:20:35.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.436+0000 7f585f7fe640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f58540a3de0 0x7f58540a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.436+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f58600d0e80 con 0x7f58681035a0 2024-04-23T18:20:35.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.436+0000 7f586659c640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f58540a3de0 0x7f58540a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.437+0000 7f586659c640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f58540a3de0 0x7f58540a62a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f585800abe0 tx=0x7f585800a2d0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.440+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5860016030 con 0x7f58681035a0 2024-04-23T18:20:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.629+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5868101a40 con 0x7f58681035a0 2024-04-23T18:20:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.630+0000 7f585f7fe640 1 -- 172.21.15.37:0/2239443925 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f586009d3c0 con 0x7f58681035a0 2024-04-23T18:20:35.624 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:20:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f58540a3de0 msgr2=0x7f58540a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f58540a3de0 0x7f58540a62a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f585800abe0 tx=0x7f585800a2d0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 msgr2=0x7f58681a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 0x7f58681a68c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f586000eaa0 tx=0x7f586000eea0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 shutdown_connections 2024-04-23T18:20:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f58540a3de0 0x7f58540a62a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58681035a0 0x7f58681a68c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5868106160 0x7f5868103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 --2- 172.21.15.37:0/2239443925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5868101090 0x7f5868102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 >> 172.21.15.37:0/2239443925 conn(0x7f5868077f50 msgr2=0x7f5868069440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 shutdown_connections 2024-04-23T18:20:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.632+0000 7f586ccc2640 1 -- 172.21.15.37:0/2239443925 wait complete. 2024-04-23T18:20:35.637 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-23T18:20:35.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:20:35.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/3613227843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 msgr2=0x7f7b0410e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/3613227843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b0410e9e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009ca0 tx=0x7f7afc02a950 comp rx=0 tx=0).stop 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/3613227843 shutdown_connections 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/3613227843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b0410e9e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/3613227843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103b50 0x7f7b0410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/3613227843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103230 0x7f7b04103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/3613227843 >> 172.21.15.37:0/3613227843 conn(0x7f7b04077fb0 msgr2=0x7f7b040783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/3613227843 shutdown_connections 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.745+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/3613227843 wait complete. 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 Processor -- start 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 -- start start 2024-04-23T18:20:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103230 0x7f7b04100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103b50 0x7f7b041013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b04101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041121b0 con 0x7f7b0410c570 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112030 con 0x7f7b04103230 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0bc2f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112330 con 0x7f7b04103b50 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0a1cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b04101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.746+0000 7f7b0a1cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b04101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42990/0 (socket says 172.21.15.37:42990) 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 -- 172.21.15.37:0/419621503 learned_addr learned my addr 172.21.15.37:0/419621503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b091ca640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103b50 0x7f7b041013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b099cb640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103230 0x7f7b04100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 -- 172.21.15.37:0/419621503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103b50 msgr2=0x7f7b041013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103b50 0x7f7b041013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 -- 172.21.15.37:0/419621503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103230 msgr2=0x7f7b04100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103230 0x7f7b04100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 -- 172.21.15.37:0/419621503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b041a6b90 con 0x7f7b0410c570 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b099cb640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103230 0x7f7b04100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0a1cc640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b04101900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7afc004af0 tx=0x7f7afc02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc034040 con 0x7f7b0410c570 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b041a6e20 con 0x7f7b0410c570 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc02fe80 con 0x7f7b0410c570 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0046a0 con 0x7f7b0410c570 2024-04-23T18:20:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.747+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a72e0 con 0x7f7b0410c570 2024-04-23T18:20:35.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.748+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b040691c0 con 0x7f7b0410c570 2024-04-23T18:20:35.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.749+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7afc037020 con 0x7f7b0410c570 2024-04-23T18:20:35.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.750+0000 7f7afaffd640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7af00a3de0 0x7f7af00a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:35.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.750+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f7afc0f16b0 con 0x7f7b0410c570 2024-04-23T18:20:35.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.750+0000 7f7b099cb640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7af00a3de0 0x7f7af00a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:35.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.751+0000 7f7b099cb640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7af00a3de0 0x7f7af00a62a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7b000058c0 tx=0x7f7b00005850 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:35.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.754+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7afc0bdd00 con 0x7f7b0410c570 2024-04-23T18:20:35.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.937+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b04062390 con 0x7f7af00a3de0 2024-04-23T18:20:35.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.945+0000 7f7afaffd640 1 -- 172.21.15.37:0/419621503 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257751 (secure 0 0 0) 0x7f7b04062390 con 0x7f7af00a3de0 2024-04-23T18:20:35.942 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:20:35.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7af00a3de0 msgr2=0x7f7af00a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7af00a3de0 0x7f7af00a62a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7b000058c0 tx=0x7f7b00005850 comp rx=0 tx=0).stop 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 msgr2=0x7f7b04101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b04101900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7afc004af0 tx=0x7f7afc02f5d0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 shutdown_connections 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7af00a3de0 0x7f7af00a62a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0410c570 0x7f7b04101900 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b04103b50 0x7f7b041013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 --2- 172.21.15.37:0/419621503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04103230 0x7f7b04100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 >> 172.21.15.37:0/419621503 conn(0x7f7b04077fb0 msgr2=0x7f7b040fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 shutdown_connections 2024-04-23T18:20:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:35.948+0000 7f7b0bc2f640 1 -- 172.21.15.37:0/419621503 wait complete. 2024-04-23T18:20:35.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:35.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/475150211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:35.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:35.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:35.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:35.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2674091914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:35 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2239443925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:35.955 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T18:20:34.571787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188241,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":926,"num_read_kb":624,"num_write":468,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1241933,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389636,"kb_used_data":15312,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552188,"statfs":{"total":1151940427776,"available":1151541440512,"internally_reserved":0,"allocated":15679488,"data_stored":9198054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990020"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109725+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:20:31.109725+0000","last_peered":"2024-04-23T18:20:31.109725+0000","last_clean":"2024-04-23T18:20:31.109725+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:20:31.109725+0000","last_undegraded":"2024-04-23T18:20:31.109725+0000","last_fullsized":"2024-04-23T18:20:31.109725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809044+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:20:30.809044+0000","last_peered":"2024-04-23T18:20:30.809044+0000","last_clean":"2024-04-23T18:20:30.809044+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:20:30.809044+0000","last_undegraded":"2024-04-23T18:20:30.809044+0000","last_fullsized":"2024-04-23T18:20:30.809044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808851+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:20:30.808851+0000","last_peered":"2024-04-23T18:20:30.808851+0000","last_clean":"2024-04-23T18:20:30.808851+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:20:30.808851+0000","last_undegraded":"2024-04-23T18:20:30.808851+0000","last_fullsized":"2024-04-23T18:20:30.808851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557782+0000","last_change":"2024-04-23T18:19:41.694639+0000","last_active":"2024-04-23T18:20:30.557782+0000","last_peered":"2024-04-23T18:20:30.557782+0000","last_clean":"2024-04-23T18:20:30.557782+0000","last_became_active":"2024-04-23T18:19:41.694291+0000","last_became_peered":"2024-04-23T18:19:41.694291+0000","last_unstale":"2024-04-23T18:20:30.557782+0000","last_undegraded":"2024-04-23T18:20:30.557782+0000","last_fullsized":"2024-04-23T18:20:30.557782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:44.268737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655753+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:20:30.655753+0000","last_peered":"2024-04-23T18:20:30.655753+0000","last_clean":"2024-04-23T18:20:30.655753+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:20:30.655753+0000","last_undegraded":"2024-04-23T18:20:30.655753+0000","last_fullsized":"2024-04-23T18:20:30.655753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665534+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:20:30.665534+0000","last_peered":"2024-04-23T18:20:30.665534+0000","last_clean":"2024-04-23T18:20:30.665534+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:20:30.665534+0000","last_undegraded":"2024-04-23T18:20:30.665534+0000","last_fullsized":"2024-04-23T18:20:30.665534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806744+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:20:30.806744+0000","last_peered":"2024-04-23T18:20:30.806744+0000","last_clean":"2024-04-23T18:20:30.806744+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:20:30.806744+0000","last_undegraded":"2024-04-23T18:20:30.806744+0000","last_fullsized":"2024-04-23T18:20:30.806744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655576+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:20:30.655576+0000","last_peered":"2024-04-23T18:20:30.655576+0000","last_clean":"2024-04-23T18:20:30.655576+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:20:30.655576+0000","last_undegraded":"2024-04-23T18:20:30.655576+0000","last_fullsized":"2024-04-23T18:20:30.655576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807041+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:20:30.807041+0000","last_peered":"2024-04-23T18:20:30.807041+0000","last_clean":"2024-04-23T18:20:30.807041+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:20:30.807041+0000","last_undegraded":"2024-04-23T18:20:30.807041+0000","last_fullsized":"2024-04-23T18:20:30.807041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665845+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:20:30.665845+0000","last_peered":"2024-04-23T18:20:30.665845+0000","last_clean":"2024-04-23T18:20:30.665845+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:20:30.665845+0000","last_undegraded":"2024-04-23T18:20:30.665845+0000","last_fullsized":"2024-04-23T18:20:30.665845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807817+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:20:30.807817+0000","last_peered":"2024-04-23T18:20:30.807817+0000","last_clean":"2024-04-23T18:20:30.807817+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:20:30.807817+0000","last_undegraded":"2024-04-23T18:20:30.807817+0000","last_fullsized":"2024-04-23T18:20:30.807817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559597+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:20:30.559597+0000","last_peered":"2024-04-23T18:20:30.559597+0000","last_clean":"2024-04-23T18:20:30.559597+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:20:30.559597+0000","last_undegraded":"2024-04-23T18:20:30.559597+0000","last_fullsized":"2024-04-23T18:20:30.559597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806683+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:20:30.806683+0000","last_peered":"2024-04-23T18:20:30.806683+0000","last_clean":"2024-04-23T18:20:30.806683+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:20:30.806683+0000","last_undegraded":"2024-04-23T18:20:30.806683+0000","last_fullsized":"2024-04-23T18:20:30.806683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808485+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:20:30.808485+0000","last_peered":"2024-04-23T18:20:30.808485+0000","last_clean":"2024-04-23T18:20:30.808485+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:20:30.808485+0000","last_undegraded":"2024-04-23T18:20:30.808485+0000","last_fullsized":"2024-04-23T18:20:30.808485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808365+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:20:30.808365+0000","last_peered":"2024-04-23T18:20:30.808365+0000","last_clean":"2024-04-23T18:20:30.808365+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:20:30.808365+0000","last_undegraded":"2024-04-23T18:20:30.808365+0000","last_fullsized":"2024-04-23T18:20:30.808365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.805909+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:20:30.805909+0000","last_peered":"2024-04-23T18:20:30.805909+0000","last_clean":"2024-04-23T18:20:30.805909+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:20:30.805909+0000","last_undegraded":"2024-04-23T18:20:30.805909+0000","last_fullsized":"2024-04-23T18:20:30.805909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109804+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:20:31.109804+0000","last_peered":"2024-04-23T18:20:31.109804+0000","last_clean":"2024-04-23T18:20:31.109804+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:20:31.109804+0000","last_undegraded":"2024-04-23T18:20:31.109804+0000","last_fullsized":"2024-04-23T18:20:31.109804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557930+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:20:30.557930+0000","last_peered":"2024-04-23T18:20:30.557930+0000","last_clean":"2024-04-23T18:20:30.557930+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:20:30.557930+0000","last_undegraded":"2024-04-23T18:20:30.557930+0000","last_fullsized":"2024-04-23T18:20:30.557930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562690+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:20:30.562690+0000","last_peered":"2024-04-23T18:20:30.562690+0000","last_clean":"2024-04-23T18:20:30.562690+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:20:30.562690+0000","last_undegraded":"2024-04-23T18:20:30.562690+0000","last_fullsized":"2024-04-23T18:20:30.562690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557974+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:20:30.557974+0000","last_peered":"2024-04-23T18:20:30.557974+0000","last_clean":"2024-04-23T18:20:30.557974+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:20:30.557974+0000","last_undegraded":"2024-04-23T18:20:30.557974+0000","last_fullsized":"2024-04-23T18:20:30.557974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656540+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:20:30.656540+0000","last_peered":"2024-04-23T18:20:30.656540+0000","last_clean":"2024-04-23T18:20:30.656540+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:20:30.656540+0000","last_undegraded":"2024-04-23T18:20:30.656540+0000","last_fullsized":"2024-04-23T18:20:30.656540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656813+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:20:30.656813+0000","last_peered":"2024-04-23T18:20:30.656813+0000","last_clean":"2024-04-23T18:20:30.656813+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:20:30.656813+0000","last_undegraded":"2024-04-23T18:20:30.656813+0000","last_fullsized":"2024-04-23T18:20:30.656813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809178+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:20:30.809178+0000","last_peered":"2024-04-23T18:20:30.809178+0000","last_clean":"2024-04-23T18:20:30.809178+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:20:30.809178+0000","last_undegraded":"2024-04-23T18:20:30.809178+0000","last_fullsized":"2024-04-23T18:20:30.809178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656091+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:20:30.656091+0000","last_peered":"2024-04-23T18:20:30.656091+0000","last_clean":"2024-04-23T18:20:30.656091+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:20:30.656091+0000","last_undegraded":"2024-04-23T18:20:30.656091+0000","last_fullsized":"2024-04-23T18:20:30.656091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806401+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:20:30.806401+0000","last_peered":"2024-04-23T18:20:30.806401+0000","last_clean":"2024-04-23T18:20:30.806401+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:20:30.806401+0000","last_undegraded":"2024-04-23T18:20:30.806401+0000","last_fullsized":"2024-04-23T18:20:30.806401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655978+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:20:30.655978+0000","last_peered":"2024-04-23T18:20:30.655978+0000","last_clean":"2024-04-23T18:20:30.655978+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:20:30.655978+0000","last_undegraded":"2024-04-23T18:20:30.655978+0000","last_fullsized":"2024-04-23T18:20:30.655978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662948+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:20:30.662948+0000","last_peered":"2024-04-23T18:20:30.662948+0000","last_clean":"2024-04-23T18:20:30.662948+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:20:30.662948+0000","last_undegraded":"2024-04-23T18:20:30.662948+0000","last_fullsized":"2024-04-23T18:20:30.662948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":144,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662757+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:20:30.662757+0000","last_peered":"2024-04-23T18:20:30.662757+0000","last_clean":"2024-04-23T18:20:30.662757+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:20:30.662757+0000","last_undegraded":"2024-04-23T18:20:30.662757+0000","last_fullsized":"2024-04-23T18:20:30.662757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655725+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:20:30.655725+0000","last_peered":"2024-04-23T18:20:30.655725+0000","last_clean":"2024-04-23T18:20:30.655725+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:20:30.655725+0000","last_undegraded":"2024-04-23T18:20:30.655725+0000","last_fullsized":"2024-04-23T18:20:30.655725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665477+0000","last_change":"2024-04-23T18:19:41.933594+0000","last_active":"2024-04-23T18:20:30.665477+0000","last_peered":"2024-04-23T18:20:30.665477+0000","last_clean":"2024-04-23T18:20:30.665477+0000","last_became_active":"2024-04-23T18:19:41.930226+0000","last_became_peered":"2024-04-23T18:19:41.930226+0000","last_unstale":"2024-04-23T18:20:30.665477+0000","last_undegraded":"2024-04-23T18:20:30.665477+0000","last_fullsized":"2024-04-23T18:20:30.665477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:57.183719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662758+0000","last_change":"2024-04-23T18:19:41.694345+0000","last_active":"2024-04-23T18:20:30.662758+0000","last_peered":"2024-04-23T18:20:30.662758+0000","last_clean":"2024-04-23T18:20:30.662758+0000","last_became_active":"2024-04-23T18:19:41.693912+0000","last_became_peered":"2024-04-23T18:19:41.693912+0000","last_unstale":"2024-04-23T18:20:30.662758+0000","last_undegraded":"2024-04-23T18:20:30.662758+0000","last_fullsized":"2024-04-23T18:20:30.662758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:12:07.153403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806576+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:20:30.806576+0000","last_peered":"2024-04-23T18:20:30.806576+0000","last_clean":"2024-04-23T18:20:30.806576+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:20:30.806576+0000","last_undegraded":"2024-04-23T18:20:30.806576+0000","last_fullsized":"2024-04-23T18:20:30.806576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655788+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:20:30.655788+0000","last_peered":"2024-04-23T18:20:30.655788+0000","last_clean":"2024-04-23T18:20:30.655788+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:20:30.655788+0000","last_undegraded":"2024-04-23T18:20:30.655788+0000","last_fullsized":"2024-04-23T18:20:30.655788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662554+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:20:30.662554+0000","last_peered":"2024-04-23T18:20:30.662554+0000","last_clean":"2024-04-23T18:20:30.662554+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:20:30.662554+0000","last_undegraded":"2024-04-23T18:20:30.662554+0000","last_fullsized":"2024-04-23T18:20:30.662554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557710+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:20:30.557710+0000","last_peered":"2024-04-23T18:20:30.557710+0000","last_clean":"2024-04-23T18:20:30.557710+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:20:30.557710+0000","last_undegraded":"2024-04-23T18:20:30.557710+0000","last_fullsized":"2024-04-23T18:20:30.557710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665821+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:20:30.665821+0000","last_peered":"2024-04-23T18:20:30.665821+0000","last_clean":"2024-04-23T18:20:30.665821+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:20:30.665821+0000","last_undegraded":"2024-04-23T18:20:30.665821+0000","last_fullsized":"2024-04-23T18:20:30.665821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806707+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:20:30.806707+0000","last_peered":"2024-04-23T18:20:30.806707+0000","last_clean":"2024-04-23T18:20:30.806707+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:20:30.806707+0000","last_undegraded":"2024-04-23T18:20:30.806707+0000","last_fullsized":"2024-04-23T18:20:30.806707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807724+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:20:30.807724+0000","last_peered":"2024-04-23T18:20:30.807724+0000","last_clean":"2024-04-23T18:20:30.807724+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:20:30.807724+0000","last_undegraded":"2024-04-23T18:20:30.807724+0000","last_fullsized":"2024-04-23T18:20:30.807724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809126+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:20:30.809126+0000","last_peered":"2024-04-23T18:20:30.809126+0000","last_clean":"2024-04-23T18:20:30.809126+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:20:30.809126+0000","last_undegraded":"2024-04-23T18:20:30.809126+0000","last_fullsized":"2024-04-23T18:20:30.809126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662978+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:20:30.662978+0000","last_peered":"2024-04-23T18:20:30.662978+0000","last_clean":"2024-04-23T18:20:30.662978+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:20:30.662978+0000","last_undegraded":"2024-04-23T18:20:30.662978+0000","last_fullsized":"2024-04-23T18:20:30.662978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806650+0000","last_change":"2024-04-23T18:19:41.694590+0000","last_active":"2024-04-23T18:20:30.806650+0000","last_peered":"2024-04-23T18:20:30.806650+0000","last_clean":"2024-04-23T18:20:30.806650+0000","last_became_active":"2024-04-23T18:19:41.694163+0000","last_became_peered":"2024-04-23T18:19:41.694163+0000","last_unstale":"2024-04-23T18:20:30.806650+0000","last_undegraded":"2024-04-23T18:20:30.806650+0000","last_fullsized":"2024-04-23T18:20:30.806650+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:28.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":308,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.007812+0000","last_change":"2024-04-23T18:19:41.940140+0000","last_active":"2024-04-23T18:20:31.007812+0000","last_peered":"2024-04-23T18:20:31.007812+0000","last_clean":"2024-04-23T18:20:31.007812+0000","last_became_active":"2024-04-23T18:19:41.931221+0000","last_became_peered":"2024-04-23T18:19:41.931221+0000","last_unstale":"2024-04-23T18:20:31.007812+0000","last_undegraded":"2024-04-23T18:20:31.007812+0000","last_fullsized":"2024-04-23T18:20:31.007812+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:40.122956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":227,"num_read_kb":179,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655901+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:20:30.655901+0000","last_peered":"2024-04-23T18:20:30.655901+0000","last_clean":"2024-04-23T18:20:30.655901+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:20:30.655901+0000","last_undegraded":"2024-04-23T18:20:30.655901+0000","last_fullsized":"2024-04-23T18:20:30.655901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109715+0000","last_change":"2024-04-23T18:19:41.699880+0000","last_active":"2024-04-23T18:20:31.109715+0000","last_peered":"2024-04-23T18:20:31.109715+0000","last_clean":"2024-04-23T18:20:31.109715+0000","last_became_active":"2024-04-23T18:19:41.691650+0000","last_became_peered":"2024-04-23T18:19:41.691650+0000","last_unstale":"2024-04-23T18:20:31.109715+0000","last_undegraded":"2024-04-23T18:20:31.109715+0000","last_fullsized":"2024-04-23T18:20:31.109715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:28.166844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806532+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:20:30.806532+0000","last_peered":"2024-04-23T18:20:30.806532+0000","last_clean":"2024-04-23T18:20:30.806532+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:20:30.806532+0000","last_undegraded":"2024-04-23T18:20:30.806532+0000","last_fullsized":"2024-04-23T18:20:30.806532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662756+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:20:30.662756+0000","last_peered":"2024-04-23T18:20:30.662756+0000","last_clean":"2024-04-23T18:20:30.662756+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:20:30.662756+0000","last_undegraded":"2024-04-23T18:20:30.662756+0000","last_fullsized":"2024-04-23T18:20:30.662756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807217+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:20:30.807217+0000","last_peered":"2024-04-23T18:20:30.807217+0000","last_clean":"2024-04-23T18:20:30.807217+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:20:30.807217+0000","last_undegraded":"2024-04-23T18:20:30.807217+0000","last_fullsized":"2024-04-23T18:20:30.807217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":144,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.260785+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:20:31.260785+0000","last_peered":"2024-04-23T18:20:31.260785+0000","last_clean":"2024-04-23T18:20:31.260785+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:20:31.260785+0000","last_undegraded":"2024-04-23T18:20:31.260785+0000","last_fullsized":"2024-04-23T18:20:31.260785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806508+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:20:30.806508+0000","last_peered":"2024-04-23T18:20:30.806508+0000","last_clean":"2024-04-23T18:20:30.806508+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:20:30.806508+0000","last_undegraded":"2024-04-23T18:20:30.806508+0000","last_fullsized":"2024-04-23T18:20:30.806508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655631+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:20:30.655631+0000","last_peered":"2024-04-23T18:20:30.655631+0000","last_clean":"2024-04-23T18:20:30.655631+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:20:30.655631+0000","last_undegraded":"2024-04-23T18:20:30.655631+0000","last_fullsized":"2024-04-23T18:20:30.655631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662693+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:20:30.662693+0000","last_peered":"2024-04-23T18:20:30.662693+0000","last_clean":"2024-04-23T18:20:30.662693+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:20:30.662693+0000","last_undegraded":"2024-04-23T18:20:30.662693+0000","last_fullsized":"2024-04-23T18:20:30.662693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656007+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:20:30.656007+0000","last_peered":"2024-04-23T18:20:30.656007+0000","last_clean":"2024-04-23T18:20:30.656007+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:20:30.656007+0000","last_undegraded":"2024-04-23T18:20:30.656007+0000","last_fullsized":"2024-04-23T18:20:30.656007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109660+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:20:31.109660+0000","last_peered":"2024-04-23T18:20:31.109660+0000","last_clean":"2024-04-23T18:20:31.109660+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:20:31.109660+0000","last_undegraded":"2024-04-23T18:20:31.109660+0000","last_fullsized":"2024-04-23T18:20:31.109660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557746+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:20:30.557746+0000","last_peered":"2024-04-23T18:20:30.557746+0000","last_clean":"2024-04-23T18:20:30.557746+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:20:30.557746+0000","last_undegraded":"2024-04-23T18:20:30.557746+0000","last_fullsized":"2024-04-23T18:20:30.557746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665629+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:20:30.665629+0000","last_peered":"2024-04-23T18:20:30.665629+0000","last_clean":"2024-04-23T18:20:30.665629+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:20:30.665629+0000","last_undegraded":"2024-04-23T18:20:30.665629+0000","last_fullsized":"2024-04-23T18:20:30.665629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807355+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:20:30.807355+0000","last_peered":"2024-04-23T18:20:30.807355+0000","last_clean":"2024-04-23T18:20:30.807355+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:20:30.807355+0000","last_undegraded":"2024-04-23T18:20:30.807355+0000","last_fullsized":"2024-04-23T18:20:30.807355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665585+0000","last_change":"2024-04-23T18:19:42.003042+0000","last_active":"2024-04-23T18:20:30.665585+0000","last_peered":"2024-04-23T18:20:30.665585+0000","last_clean":"2024-04-23T18:20:30.665585+0000","last_became_active":"2024-04-23T18:19:42.002728+0000","last_became_peered":"2024-04-23T18:19:42.002728+0000","last_unstale":"2024-04-23T18:20:30.665585+0000","last_undegraded":"2024-04-23T18:20:30.665585+0000","last_fullsized":"2024-04-23T18:20:30.665585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:54:33.401918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662782+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:20:30.662782+0000","last_peered":"2024-04-23T18:20:30.662782+0000","last_clean":"2024-04-23T18:20:30.662782+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:20:30.662782+0000","last_undegraded":"2024-04-23T18:20:30.662782+0000","last_fullsized":"2024-04-23T18:20:30.662782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808223+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:20:30.808223+0000","last_peered":"2024-04-23T18:20:30.808223+0000","last_clean":"2024-04-23T18:20:30.808223+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:20:30.808223+0000","last_undegraded":"2024-04-23T18:20:30.808223+0000","last_fullsized":"2024-04-23T18:20:30.808223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656274+0000","last_change":"2024-04-23T18:19:41.691129+0000","last_active":"2024-04-23T18:20:30.656274+0000","last_peered":"2024-04-23T18:20:30.656274+0000","last_clean":"2024-04-23T18:20:30.656274+0000","last_became_active":"2024-04-23T18:19:41.690710+0000","last_became_peered":"2024-04-23T18:19:41.690710+0000","last_unstale":"2024-04-23T18:20:30.656274+0000","last_undegraded":"2024-04-23T18:20:30.656274+0000","last_fullsized":"2024-04-23T18:20:30.656274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:51.914794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559388+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:20:30.559388+0000","last_peered":"2024-04-23T18:20:30.559388+0000","last_clean":"2024-04-23T18:20:30.559388+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:20:30.559388+0000","last_undegraded":"2024-04-23T18:20:30.559388+0000","last_fullsized":"2024-04-23T18:20:30.559388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806758+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:20:30.806758+0000","last_peered":"2024-04-23T18:20:30.806758+0000","last_clean":"2024-04-23T18:20:30.806758+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:20:30.806758+0000","last_undegraded":"2024-04-23T18:20:30.806758+0000","last_fullsized":"2024-04-23T18:20:30.806758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806380+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:20:30.806380+0000","last_peered":"2024-04-23T18:20:30.806380+0000","last_clean":"2024-04-23T18:20:30.806380+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:20:30.806380+0000","last_undegraded":"2024-04-23T18:20:30.806380+0000","last_fullsized":"2024-04-23T18:20:30.806380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806975+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:20:30.806975+0000","last_peered":"2024-04-23T18:20:30.806975+0000","last_clean":"2024-04-23T18:20:30.806975+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:20:30.806975+0000","last_undegraded":"2024-04-23T18:20:30.806975+0000","last_fullsized":"2024-04-23T18:20:30.806975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655969+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:20:30.655969+0000","last_peered":"2024-04-23T18:20:30.655969+0000","last_clean":"2024-04-23T18:20:30.655969+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:20:30.655969+0000","last_undegraded":"2024-04-23T18:20:30.655969+0000","last_fullsized":"2024-04-23T18:20:30.655969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806739+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:20:30.806739+0000","last_peered":"2024-04-23T18:20:30.806739+0000","last_clean":"2024-04-23T18:20:30.806739+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:20:30.806739+0000","last_undegraded":"2024-04-23T18:20:30.806739+0000","last_fullsized":"2024-04-23T18:20:30.806739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'45","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.774689+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:20:30.774689+0000","last_peered":"2024-04-23T18:20:30.774689+0000","last_clean":"2024-04-23T18:20:30.774689+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:20:30.774689+0000","last_undegraded":"2024-04-23T18:20:30.774689+0000","last_fullsized":"2024-04-23T18:20:30.774689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656706+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:20:30.656706+0000","last_peered":"2024-04-23T18:20:30.656706+0000","last_clean":"2024-04-23T18:20:30.656706+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:20:30.656706+0000","last_undegraded":"2024-04-23T18:20:30.656706+0000","last_fullsized":"2024-04-23T18:20:30.656706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559381+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:20:30.559381+0000","last_peered":"2024-04-23T18:20:30.559381+0000","last_clean":"2024-04-23T18:20:30.559381+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:20:30.559381+0000","last_undegraded":"2024-04-23T18:20:30.559381+0000","last_fullsized":"2024-04-23T18:20:30.559381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809178+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:20:30.809178+0000","last_peered":"2024-04-23T18:20:30.809178+0000","last_clean":"2024-04-23T18:20:30.809178+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:20:30.809178+0000","last_undegraded":"2024-04-23T18:20:30.809178+0000","last_fullsized":"2024-04-23T18:20:30.809178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806198+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:20:30.806198+0000","last_peered":"2024-04-23T18:20:30.806198+0000","last_clean":"2024-04-23T18:20:30.806198+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:20:30.806198+0000","last_undegraded":"2024-04-23T18:20:30.806198+0000","last_fullsized":"2024-04-23T18:20:30.806198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655622+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:20:30.655622+0000","last_peered":"2024-04-23T18:20:30.655622+0000","last_clean":"2024-04-23T18:20:30.655622+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:20:30.655622+0000","last_undegraded":"2024-04-23T18:20:30.655622+0000","last_fullsized":"2024-04-23T18:20:30.655622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655976+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:20:30.655976+0000","last_peered":"2024-04-23T18:20:30.655976+0000","last_clean":"2024-04-23T18:20:30.655976+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:20:30.655976+0000","last_undegraded":"2024-04-23T18:20:30.655976+0000","last_fullsized":"2024-04-23T18:20:30.655976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109391+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:20:31.109391+0000","last_peered":"2024-04-23T18:20:31.109391+0000","last_clean":"2024-04-23T18:20:31.109391+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:20:31.109391+0000","last_undegraded":"2024-04-23T18:20:31.109391+0000","last_fullsized":"2024-04-23T18:20:31.109391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562843+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:20:30.562843+0000","last_peered":"2024-04-23T18:20:30.562843+0000","last_clean":"2024-04-23T18:20:30.562843+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:20:30.562843+0000","last_undegraded":"2024-04-23T18:20:30.562843+0000","last_fullsized":"2024-04-23T18:20:30.562843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655954+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:20:30.655954+0000","last_peered":"2024-04-23T18:20:30.655954+0000","last_clean":"2024-04-23T18:20:30.655954+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:20:30.655954+0000","last_undegraded":"2024-04-23T18:20:30.655954+0000","last_fullsized":"2024-04-23T18:20:30.655954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655726+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:20:30.655726+0000","last_peered":"2024-04-23T18:20:30.655726+0000","last_clean":"2024-04-23T18:20:30.655726+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:20:30.655726+0000","last_undegraded":"2024-04-23T18:20:30.655726+0000","last_fullsized":"2024-04-23T18:20:30.655726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807464+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:20:30.807464+0000","last_peered":"2024-04-23T18:20:30.807464+0000","last_clean":"2024-04-23T18:20:30.807464+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:20:30.807464+0000","last_undegraded":"2024-04-23T18:20:30.807464+0000","last_fullsized":"2024-04-23T18:20:30.807464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562850+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:20:30.562850+0000","last_peered":"2024-04-23T18:20:30.562850+0000","last_clean":"2024-04-23T18:20:30.562850+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:20:30.562850+0000","last_undegraded":"2024-04-23T18:20:30.562850+0000","last_fullsized":"2024-04-23T18:20:30.562850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807721+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:20:30.807721+0000","last_peered":"2024-04-23T18:20:30.807721+0000","last_clean":"2024-04-23T18:20:30.807721+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:20:30.807721+0000","last_undegraded":"2024-04-23T18:20:30.807721+0000","last_fullsized":"2024-04-23T18:20:30.807721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.770167+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:20:30.770167+0000","last_peered":"2024-04-23T18:20:30.770167+0000","last_clean":"2024-04-23T18:20:30.770167+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:20:30.770167+0000","last_undegraded":"2024-04-23T18:20:30.770167+0000","last_fullsized":"2024-04-23T18:20:30.770167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807610+0000","last_change":"2024-04-23T18:19:41.698372+0000","last_active":"2024-04-23T18:20:30.807610+0000","last_peered":"2024-04-23T18:20:30.807610+0000","last_clean":"2024-04-23T18:20:30.807610+0000","last_became_active":"2024-04-23T18:19:41.695014+0000","last_became_peered":"2024-04-23T18:19:41.695014+0000","last_unstale":"2024-04-23T18:20:30.807610+0000","last_undegraded":"2024-04-23T18:20:30.807610+0000","last_fullsized":"2024-04-23T18:20:30.807610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:35:15.568241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806730+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:20:30.806730+0000","last_peered":"2024-04-23T18:20:30.806730+0000","last_clean":"2024-04-23T18:20:30.806730+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:20:30.806730+0000","last_undegraded":"2024-04-23T18:20:30.806730+0000","last_fullsized":"2024-04-23T18:20:30.806730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807869+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:20:30.807869+0000","last_peered":"2024-04-23T18:20:30.807869+0000","last_clean":"2024-04-23T18:20:30.807869+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:20:30.807869+0000","last_undegraded":"2024-04-23T18:20:30.807869+0000","last_fullsized":"2024-04-23T18:20:30.807869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109472+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:20:31.109472+0000","last_peered":"2024-04-23T18:20:31.109472+0000","last_clean":"2024-04-23T18:20:31.109472+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:20:31.109472+0000","last_undegraded":"2024-04-23T18:20:31.109472+0000","last_fullsized":"2024-04-23T18:20:31.109472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562560+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:20:30.562560+0000","last_peered":"2024-04-23T18:20:30.562560+0000","last_clean":"2024-04-23T18:20:30.562560+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:20:30.562560+0000","last_undegraded":"2024-04-23T18:20:30.562560+0000","last_fullsized":"2024-04-23T18:20:30.562560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562670+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:20:30.562670+0000","last_peered":"2024-04-23T18:20:30.562670+0000","last_clean":"2024-04-23T18:20:30.562670+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:20:30.562670+0000","last_undegraded":"2024-04-23T18:20:30.562670+0000","last_fullsized":"2024-04-23T18:20:30.562670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808727+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:20:30.808727+0000","last_peered":"2024-04-23T18:20:30.808727+0000","last_clean":"2024-04-23T18:20:30.808727+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:20:30.808727+0000","last_undegraded":"2024-04-23T18:20:30.808727+0000","last_fullsized":"2024-04-23T18:20:30.808727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806849+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:20:30.806849+0000","last_peered":"2024-04-23T18:20:30.806849+0000","last_clean":"2024-04-23T18:20:30.806849+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:20:30.806849+0000","last_undegraded":"2024-04-23T18:20:30.806849+0000","last_fullsized":"2024-04-23T18:20:30.806849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109541+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:20:31.109541+0000","last_peered":"2024-04-23T18:20:31.109541+0000","last_clean":"2024-04-23T18:20:31.109541+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:20:31.109541+0000","last_undegraded":"2024-04-23T18:20:31.109541+0000","last_fullsized":"2024-04-23T18:20:31.109541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562953+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:20:30.562953+0000","last_peered":"2024-04-23T18:20:30.562953+0000","last_clean":"2024-04-23T18:20:30.562953+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:20:30.562953+0000","last_undegraded":"2024-04-23T18:20:30.562953+0000","last_fullsized":"2024-04-23T18:20:30.562953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656683+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:20:30.656683+0000","last_peered":"2024-04-23T18:20:30.656683+0000","last_clean":"2024-04-23T18:20:30.656683+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:20:30.656683+0000","last_undegraded":"2024-04-23T18:20:30.656683+0000","last_fullsized":"2024-04-23T18:20:30.656683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662620+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:20:30.662620+0000","last_peered":"2024-04-23T18:20:30.662620+0000","last_clean":"2024-04-23T18:20:30.662620+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:20:30.662620+0000","last_undegraded":"2024-04-23T18:20:30.662620+0000","last_fullsized":"2024-04-23T18:20:30.662620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655927+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:20:30.655927+0000","last_peered":"2024-04-23T18:20:30.655927+0000","last_clean":"2024-04-23T18:20:30.655927+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:20:30.655927+0000","last_undegraded":"2024-04-23T18:20:30.655927+0000","last_fullsized":"2024-04-23T18:20:30.655927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806424+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:20:30.806424+0000","last_peered":"2024-04-23T18:20:30.806424+0000","last_clean":"2024-04-23T18:20:30.806424+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:20:30.806424+0000","last_undegraded":"2024-04-23T18:20:30.806424+0000","last_fullsized":"2024-04-23T18:20:30.806424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806659+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:20:30.806659+0000","last_peered":"2024-04-23T18:20:30.806659+0000","last_clean":"2024-04-23T18:20:30.806659+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:20:30.806659+0000","last_undegraded":"2024-04-23T18:20:30.806659+0000","last_fullsized":"2024-04-23T18:20:30.806659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109535+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:20:31.109535+0000","last_peered":"2024-04-23T18:20:31.109535+0000","last_clean":"2024-04-23T18:20:31.109535+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:20:31.109535+0000","last_undegraded":"2024-04-23T18:20:31.109535+0000","last_fullsized":"2024-04-23T18:20:31.109535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597639,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":790,"num_read_kb":505,"num_write":285,"num_write_kb":1436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":61080,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1792917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285603,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":472865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1712128,"data_stored":1141296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1417216,"data_stored":875364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005107,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1417216,"data_stored":875284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1613824,"data_stored":1074945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":481420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":548971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":482691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1006430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":550538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37232,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707920,"statfs":{"total":95995035648,"available":95956910080,"internally_reserved":0,"allocated":1757184,"data_stored":1205189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:20:35.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:20:36.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/475150211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2674091914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:36.089 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:35 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2239443925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 -- 172.21.15.37:0/4075786098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c102240 msgr2=0x7f279c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 --2- 172.21.15.37:0/4075786098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c102240 0x7f279c102620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 -- 172.21.15.37:0/4075786098 shutdown_connections 2024-04-23T18:20:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 --2- 172.21.15.37:0/4075786098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279c107280 0x7f279c10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 --2- 172.21.15.37:0/4075786098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f279c102bf0 0x7f279c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 --2- 172.21.15.37:0/4075786098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c102240 0x7f279c102620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 -- 172.21.15.37:0/4075786098 >> 172.21.15.37:0/4075786098 conn(0x7f279c0790e0 msgr2=0x7f279c0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 -- 172.21.15.37:0/4075786098 shutdown_connections 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.103+0000 7f27a31f7640 1 -- 172.21.15.37:0/4075786098 wait complete. 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 Processor -- start 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 -- start start 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f279c102240 0x7f279c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 0x7f279c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279c074cd0 0x7f279c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f279c112200 con 0x7f279c107280 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f279c112080 con 0x7f279c074cd0 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a31f7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f279c112380 con 0x7f279c102240 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 0x7f279c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 0x7f279c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43010/0 (socket says 172.21.15.37:43010) 2024-04-23T18:20:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 -- 172.21.15.37:0/408255989 learned_addr learned my addr 172.21.15.37:0/408255989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f27a0f93640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f279c102240 0x7f279c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 -- 172.21.15.37:0/408255989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f279c102240 msgr2=0x7f279c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f279c102240 0x7f279c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 -- 172.21.15.37:0/408255989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279c074cd0 msgr2=0x7f279c1a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279c074cd0 0x7f279c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.104+0000 7f279bfff640 1 -- 172.21.15.37:0/408255989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c1a7030 con 0x7f279c107280 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.105+0000 7f279bfff640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 0x7f279c074790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f278400e9c0 tx=0x7f278400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.105+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278400ccc0 con 0x7f279c107280 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.105+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c1a7320 con 0x7f279c107280 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.105+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27840054d0 con 0x7f279c107280 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.105+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784010430 con 0x7f279c107280 2024-04-23T18:20:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.105+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c1a7870 con 0x7f279c107280 2024-04-23T18:20:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.106+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279c0691c0 con 0x7f279c107280 2024-04-23T18:20:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.107+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f2784005020 con 0x7f279c107280 2024-04-23T18:20:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.108+0000 7f2799ffb640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27880a3de0 0x7f27880a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.108+0000 7f27a0f93640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27880a3de0 0x7f27880a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.108+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f27840d0e10 con 0x7f279c107280 2024-04-23T18:20:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.108+0000 7f27a0f93640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27880a3de0 0x7f27880a62a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f278c006010 tx=0x7f278c000bc0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: Updating smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: Updating smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: Updating smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:20:36.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/475150211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:20:36.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2674091914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:36.107 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:35 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2239443925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.112+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f278409d350 con 0x7f279c107280 2024-04-23T18:20:36.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.296+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f279c102bf0 con 0x7f27880a3de0 2024-04-23T18:20:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.303+0000 7f2799ffb640 1 -- 172.21.15.37:0/408255989 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257751 (secure 0 0 0) 0x7f279c102bf0 con 0x7f27880a3de0 2024-04-23T18:20:36.300 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:20:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27880a3de0 msgr2=0x7f27880a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27880a3de0 0x7f27880a62a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f278c006010 tx=0x7f278c000bc0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 msgr2=0x7f279c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 0x7f279c074790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f278400e9c0 tx=0x7f278400ee80 comp rx=0 tx=0).stop 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 shutdown_connections 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27880a3de0 0x7f27880a62a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279c074cd0 0x7f279c1a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f279c107280 0x7f279c074790 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 --2- 172.21.15.37:0/408255989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f279c102240 0x7f279c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 >> 172.21.15.37:0/408255989 conn(0x7f279c0790e0 msgr2=0x7f279c1003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 shutdown_connections 2024-04-23T18:20:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.306+0000 7f27a31f7640 1 -- 172.21.15.37:0/408255989 wait complete. 2024-04-23T18:20:36.311 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T18:20:34.571787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188241,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":926,"num_read_kb":624,"num_write":468,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1241933,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":864,"ondisk_log_size":864,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389636,"kb_used_data":15312,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552188,"statfs":{"total":1151940427776,"available":1151541440512,"internally_reserved":0,"allocated":15679488,"data_stored":9198054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990020"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109725+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:20:31.109725+0000","last_peered":"2024-04-23T18:20:31.109725+0000","last_clean":"2024-04-23T18:20:31.109725+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:20:31.109725+0000","last_undegraded":"2024-04-23T18:20:31.109725+0000","last_fullsized":"2024-04-23T18:20:31.109725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809044+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:20:30.809044+0000","last_peered":"2024-04-23T18:20:30.809044+0000","last_clean":"2024-04-23T18:20:30.809044+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:20:30.809044+0000","last_undegraded":"2024-04-23T18:20:30.809044+0000","last_fullsized":"2024-04-23T18:20:30.809044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808851+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:20:30.808851+0000","last_peered":"2024-04-23T18:20:30.808851+0000","last_clean":"2024-04-23T18:20:30.808851+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:20:30.808851+0000","last_undegraded":"2024-04-23T18:20:30.808851+0000","last_fullsized":"2024-04-23T18:20:30.808851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557782+0000","last_change":"2024-04-23T18:19:41.694639+0000","last_active":"2024-04-23T18:20:30.557782+0000","last_peered":"2024-04-23T18:20:30.557782+0000","last_clean":"2024-04-23T18:20:30.557782+0000","last_became_active":"2024-04-23T18:19:41.694291+0000","last_became_peered":"2024-04-23T18:19:41.694291+0000","last_unstale":"2024-04-23T18:20:30.557782+0000","last_undegraded":"2024-04-23T18:20:30.557782+0000","last_fullsized":"2024-04-23T18:20:30.557782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:44.268737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655753+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:20:30.655753+0000","last_peered":"2024-04-23T18:20:30.655753+0000","last_clean":"2024-04-23T18:20:30.655753+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:20:30.655753+0000","last_undegraded":"2024-04-23T18:20:30.655753+0000","last_fullsized":"2024-04-23T18:20:30.655753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665534+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:20:30.665534+0000","last_peered":"2024-04-23T18:20:30.665534+0000","last_clean":"2024-04-23T18:20:30.665534+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:20:30.665534+0000","last_undegraded":"2024-04-23T18:20:30.665534+0000","last_fullsized":"2024-04-23T18:20:30.665534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806744+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:20:30.806744+0000","last_peered":"2024-04-23T18:20:30.806744+0000","last_clean":"2024-04-23T18:20:30.806744+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:20:30.806744+0000","last_undegraded":"2024-04-23T18:20:30.806744+0000","last_fullsized":"2024-04-23T18:20:30.806744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655576+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:20:30.655576+0000","last_peered":"2024-04-23T18:20:30.655576+0000","last_clean":"2024-04-23T18:20:30.655576+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:20:30.655576+0000","last_undegraded":"2024-04-23T18:20:30.655576+0000","last_fullsized":"2024-04-23T18:20:30.655576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807041+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:20:30.807041+0000","last_peered":"2024-04-23T18:20:30.807041+0000","last_clean":"2024-04-23T18:20:30.807041+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:20:30.807041+0000","last_undegraded":"2024-04-23T18:20:30.807041+0000","last_fullsized":"2024-04-23T18:20:30.807041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665845+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:20:30.665845+0000","last_peered":"2024-04-23T18:20:30.665845+0000","last_clean":"2024-04-23T18:20:30.665845+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:20:30.665845+0000","last_undegraded":"2024-04-23T18:20:30.665845+0000","last_fullsized":"2024-04-23T18:20:30.665845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807817+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:20:30.807817+0000","last_peered":"2024-04-23T18:20:30.807817+0000","last_clean":"2024-04-23T18:20:30.807817+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:20:30.807817+0000","last_undegraded":"2024-04-23T18:20:30.807817+0000","last_fullsized":"2024-04-23T18:20:30.807817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559597+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:20:30.559597+0000","last_peered":"2024-04-23T18:20:30.559597+0000","last_clean":"2024-04-23T18:20:30.559597+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:20:30.559597+0000","last_undegraded":"2024-04-23T18:20:30.559597+0000","last_fullsized":"2024-04-23T18:20:30.559597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806683+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:20:30.806683+0000","last_peered":"2024-04-23T18:20:30.806683+0000","last_clean":"2024-04-23T18:20:30.806683+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:20:30.806683+0000","last_undegraded":"2024-04-23T18:20:30.806683+0000","last_fullsized":"2024-04-23T18:20:30.806683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808485+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:20:30.808485+0000","last_peered":"2024-04-23T18:20:30.808485+0000","last_clean":"2024-04-23T18:20:30.808485+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:20:30.808485+0000","last_undegraded":"2024-04-23T18:20:30.808485+0000","last_fullsized":"2024-04-23T18:20:30.808485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808365+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:20:30.808365+0000","last_peered":"2024-04-23T18:20:30.808365+0000","last_clean":"2024-04-23T18:20:30.808365+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:20:30.808365+0000","last_undegraded":"2024-04-23T18:20:30.808365+0000","last_fullsized":"2024-04-23T18:20:30.808365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.805909+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:20:30.805909+0000","last_peered":"2024-04-23T18:20:30.805909+0000","last_clean":"2024-04-23T18:20:30.805909+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:20:30.805909+0000","last_undegraded":"2024-04-23T18:20:30.805909+0000","last_fullsized":"2024-04-23T18:20:30.805909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109804+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:20:31.109804+0000","last_peered":"2024-04-23T18:20:31.109804+0000","last_clean":"2024-04-23T18:20:31.109804+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:20:31.109804+0000","last_undegraded":"2024-04-23T18:20:31.109804+0000","last_fullsized":"2024-04-23T18:20:31.109804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557930+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:20:30.557930+0000","last_peered":"2024-04-23T18:20:30.557930+0000","last_clean":"2024-04-23T18:20:30.557930+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:20:30.557930+0000","last_undegraded":"2024-04-23T18:20:30.557930+0000","last_fullsized":"2024-04-23T18:20:30.557930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562690+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:20:30.562690+0000","last_peered":"2024-04-23T18:20:30.562690+0000","last_clean":"2024-04-23T18:20:30.562690+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:20:30.562690+0000","last_undegraded":"2024-04-23T18:20:30.562690+0000","last_fullsized":"2024-04-23T18:20:30.562690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557974+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:20:30.557974+0000","last_peered":"2024-04-23T18:20:30.557974+0000","last_clean":"2024-04-23T18:20:30.557974+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:20:30.557974+0000","last_undegraded":"2024-04-23T18:20:30.557974+0000","last_fullsized":"2024-04-23T18:20:30.557974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656540+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:20:30.656540+0000","last_peered":"2024-04-23T18:20:30.656540+0000","last_clean":"2024-04-23T18:20:30.656540+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:20:30.656540+0000","last_undegraded":"2024-04-23T18:20:30.656540+0000","last_fullsized":"2024-04-23T18:20:30.656540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656813+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:20:30.656813+0000","last_peered":"2024-04-23T18:20:30.656813+0000","last_clean":"2024-04-23T18:20:30.656813+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:20:30.656813+0000","last_undegraded":"2024-04-23T18:20:30.656813+0000","last_fullsized":"2024-04-23T18:20:30.656813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809178+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:20:30.809178+0000","last_peered":"2024-04-23T18:20:30.809178+0000","last_clean":"2024-04-23T18:20:30.809178+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:20:30.809178+0000","last_undegraded":"2024-04-23T18:20:30.809178+0000","last_fullsized":"2024-04-23T18:20:30.809178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656091+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:20:30.656091+0000","last_peered":"2024-04-23T18:20:30.656091+0000","last_clean":"2024-04-23T18:20:30.656091+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:20:30.656091+0000","last_undegraded":"2024-04-23T18:20:30.656091+0000","last_fullsized":"2024-04-23T18:20:30.656091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806401+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:20:30.806401+0000","last_peered":"2024-04-23T18:20:30.806401+0000","last_clean":"2024-04-23T18:20:30.806401+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:20:30.806401+0000","last_undegraded":"2024-04-23T18:20:30.806401+0000","last_fullsized":"2024-04-23T18:20:30.806401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655978+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:20:30.655978+0000","last_peered":"2024-04-23T18:20:30.655978+0000","last_clean":"2024-04-23T18:20:30.655978+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:20:30.655978+0000","last_undegraded":"2024-04-23T18:20:30.655978+0000","last_fullsized":"2024-04-23T18:20:30.655978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662948+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:20:30.662948+0000","last_peered":"2024-04-23T18:20:30.662948+0000","last_clean":"2024-04-23T18:20:30.662948+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:20:30.662948+0000","last_undegraded":"2024-04-23T18:20:30.662948+0000","last_fullsized":"2024-04-23T18:20:30.662948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":144,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662757+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:20:30.662757+0000","last_peered":"2024-04-23T18:20:30.662757+0000","last_clean":"2024-04-23T18:20:30.662757+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:20:30.662757+0000","last_undegraded":"2024-04-23T18:20:30.662757+0000","last_fullsized":"2024-04-23T18:20:30.662757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655725+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:20:30.655725+0000","last_peered":"2024-04-23T18:20:30.655725+0000","last_clean":"2024-04-23T18:20:30.655725+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:20:30.655725+0000","last_undegraded":"2024-04-23T18:20:30.655725+0000","last_fullsized":"2024-04-23T18:20:30.655725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665477+0000","last_change":"2024-04-23T18:19:41.933594+0000","last_active":"2024-04-23T18:20:30.665477+0000","last_peered":"2024-04-23T18:20:30.665477+0000","last_clean":"2024-04-23T18:20:30.665477+0000","last_became_active":"2024-04-23T18:19:41.930226+0000","last_became_peered":"2024-04-23T18:19:41.930226+0000","last_unstale":"2024-04-23T18:20:30.665477+0000","last_undegraded":"2024-04-23T18:20:30.665477+0000","last_fullsized":"2024-04-23T18:20:30.665477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:57.183719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662758+0000","last_change":"2024-04-23T18:19:41.694345+0000","last_active":"2024-04-23T18:20:30.662758+0000","last_peered":"2024-04-23T18:20:30.662758+0000","last_clean":"2024-04-23T18:20:30.662758+0000","last_became_active":"2024-04-23T18:19:41.693912+0000","last_became_peered":"2024-04-23T18:19:41.693912+0000","last_unstale":"2024-04-23T18:20:30.662758+0000","last_undegraded":"2024-04-23T18:20:30.662758+0000","last_fullsized":"2024-04-23T18:20:30.662758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:12:07.153403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806576+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:20:30.806576+0000","last_peered":"2024-04-23T18:20:30.806576+0000","last_clean":"2024-04-23T18:20:30.806576+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:20:30.806576+0000","last_undegraded":"2024-04-23T18:20:30.806576+0000","last_fullsized":"2024-04-23T18:20:30.806576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655788+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:20:30.655788+0000","last_peered":"2024-04-23T18:20:30.655788+0000","last_clean":"2024-04-23T18:20:30.655788+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:20:30.655788+0000","last_undegraded":"2024-04-23T18:20:30.655788+0000","last_fullsized":"2024-04-23T18:20:30.655788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662554+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:20:30.662554+0000","last_peered":"2024-04-23T18:20:30.662554+0000","last_clean":"2024-04-23T18:20:30.662554+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:20:30.662554+0000","last_undegraded":"2024-04-23T18:20:30.662554+0000","last_fullsized":"2024-04-23T18:20:30.662554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557710+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:20:30.557710+0000","last_peered":"2024-04-23T18:20:30.557710+0000","last_clean":"2024-04-23T18:20:30.557710+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:20:30.557710+0000","last_undegraded":"2024-04-23T18:20:30.557710+0000","last_fullsized":"2024-04-23T18:20:30.557710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665821+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:20:30.665821+0000","last_peered":"2024-04-23T18:20:30.665821+0000","last_clean":"2024-04-23T18:20:30.665821+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:20:30.665821+0000","last_undegraded":"2024-04-23T18:20:30.665821+0000","last_fullsized":"2024-04-23T18:20:30.665821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806707+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:20:30.806707+0000","last_peered":"2024-04-23T18:20:30.806707+0000","last_clean":"2024-04-23T18:20:30.806707+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:20:30.806707+0000","last_undegraded":"2024-04-23T18:20:30.806707+0000","last_fullsized":"2024-04-23T18:20:30.806707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807724+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:20:30.807724+0000","last_peered":"2024-04-23T18:20:30.807724+0000","last_clean":"2024-04-23T18:20:30.807724+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:20:30.807724+0000","last_undegraded":"2024-04-23T18:20:30.807724+0000","last_fullsized":"2024-04-23T18:20:30.807724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809126+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:20:30.809126+0000","last_peered":"2024-04-23T18:20:30.809126+0000","last_clean":"2024-04-23T18:20:30.809126+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:20:30.809126+0000","last_undegraded":"2024-04-23T18:20:30.809126+0000","last_fullsized":"2024-04-23T18:20:30.809126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662978+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:20:30.662978+0000","last_peered":"2024-04-23T18:20:30.662978+0000","last_clean":"2024-04-23T18:20:30.662978+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:20:30.662978+0000","last_undegraded":"2024-04-23T18:20:30.662978+0000","last_fullsized":"2024-04-23T18:20:30.662978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806650+0000","last_change":"2024-04-23T18:19:41.694590+0000","last_active":"2024-04-23T18:20:30.806650+0000","last_peered":"2024-04-23T18:20:30.806650+0000","last_clean":"2024-04-23T18:20:30.806650+0000","last_became_active":"2024-04-23T18:19:41.694163+0000","last_became_peered":"2024-04-23T18:19:41.694163+0000","last_unstale":"2024-04-23T18:20:30.806650+0000","last_undegraded":"2024-04-23T18:20:30.806650+0000","last_fullsized":"2024-04-23T18:20:30.806650+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:28.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":308,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.007812+0000","last_change":"2024-04-23T18:19:41.940140+0000","last_active":"2024-04-23T18:20:31.007812+0000","last_peered":"2024-04-23T18:20:31.007812+0000","last_clean":"2024-04-23T18:20:31.007812+0000","last_became_active":"2024-04-23T18:19:41.931221+0000","last_became_peered":"2024-04-23T18:19:41.931221+0000","last_unstale":"2024-04-23T18:20:31.007812+0000","last_undegraded":"2024-04-23T18:20:31.007812+0000","last_fullsized":"2024-04-23T18:20:31.007812+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:40.122956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":227,"num_read_kb":179,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655901+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:20:30.655901+0000","last_peered":"2024-04-23T18:20:30.655901+0000","last_clean":"2024-04-23T18:20:30.655901+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:20:30.655901+0000","last_undegraded":"2024-04-23T18:20:30.655901+0000","last_fullsized":"2024-04-23T18:20:30.655901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109715+0000","last_change":"2024-04-23T18:19:41.699880+0000","last_active":"2024-04-23T18:20:31.109715+0000","last_peered":"2024-04-23T18:20:31.109715+0000","last_clean":"2024-04-23T18:20:31.109715+0000","last_became_active":"2024-04-23T18:19:41.691650+0000","last_became_peered":"2024-04-23T18:19:41.691650+0000","last_unstale":"2024-04-23T18:20:31.109715+0000","last_undegraded":"2024-04-23T18:20:31.109715+0000","last_fullsized":"2024-04-23T18:20:31.109715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:28.166844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806532+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:20:30.806532+0000","last_peered":"2024-04-23T18:20:30.806532+0000","last_clean":"2024-04-23T18:20:30.806532+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:20:30.806532+0000","last_undegraded":"2024-04-23T18:20:30.806532+0000","last_fullsized":"2024-04-23T18:20:30.806532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662756+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:20:30.662756+0000","last_peered":"2024-04-23T18:20:30.662756+0000","last_clean":"2024-04-23T18:20:30.662756+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:20:30.662756+0000","last_undegraded":"2024-04-23T18:20:30.662756+0000","last_fullsized":"2024-04-23T18:20:30.662756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807217+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:20:30.807217+0000","last_peered":"2024-04-23T18:20:30.807217+0000","last_clean":"2024-04-23T18:20:30.807217+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:20:30.807217+0000","last_undegraded":"2024-04-23T18:20:30.807217+0000","last_fullsized":"2024-04-23T18:20:30.807217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":144,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.260785+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:20:31.260785+0000","last_peered":"2024-04-23T18:20:31.260785+0000","last_clean":"2024-04-23T18:20:31.260785+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:20:31.260785+0000","last_undegraded":"2024-04-23T18:20:31.260785+0000","last_fullsized":"2024-04-23T18:20:31.260785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806508+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:20:30.806508+0000","last_peered":"2024-04-23T18:20:30.806508+0000","last_clean":"2024-04-23T18:20:30.806508+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:20:30.806508+0000","last_undegraded":"2024-04-23T18:20:30.806508+0000","last_fullsized":"2024-04-23T18:20:30.806508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655631+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:20:30.655631+0000","last_peered":"2024-04-23T18:20:30.655631+0000","last_clean":"2024-04-23T18:20:30.655631+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:20:30.655631+0000","last_undegraded":"2024-04-23T18:20:30.655631+0000","last_fullsized":"2024-04-23T18:20:30.655631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662693+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:20:30.662693+0000","last_peered":"2024-04-23T18:20:30.662693+0000","last_clean":"2024-04-23T18:20:30.662693+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:20:30.662693+0000","last_undegraded":"2024-04-23T18:20:30.662693+0000","last_fullsized":"2024-04-23T18:20:30.662693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656007+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:20:30.656007+0000","last_peered":"2024-04-23T18:20:30.656007+0000","last_clean":"2024-04-23T18:20:30.656007+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:20:30.656007+0000","last_undegraded":"2024-04-23T18:20:30.656007+0000","last_fullsized":"2024-04-23T18:20:30.656007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109660+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:20:31.109660+0000","last_peered":"2024-04-23T18:20:31.109660+0000","last_clean":"2024-04-23T18:20:31.109660+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:20:31.109660+0000","last_undegraded":"2024-04-23T18:20:31.109660+0000","last_fullsized":"2024-04-23T18:20:31.109660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557746+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:20:30.557746+0000","last_peered":"2024-04-23T18:20:30.557746+0000","last_clean":"2024-04-23T18:20:30.557746+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:20:30.557746+0000","last_undegraded":"2024-04-23T18:20:30.557746+0000","last_fullsized":"2024-04-23T18:20:30.557746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665629+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:20:30.665629+0000","last_peered":"2024-04-23T18:20:30.665629+0000","last_clean":"2024-04-23T18:20:30.665629+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:20:30.665629+0000","last_undegraded":"2024-04-23T18:20:30.665629+0000","last_fullsized":"2024-04-23T18:20:30.665629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807355+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:20:30.807355+0000","last_peered":"2024-04-23T18:20:30.807355+0000","last_clean":"2024-04-23T18:20:30.807355+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:20:30.807355+0000","last_undegraded":"2024-04-23T18:20:30.807355+0000","last_fullsized":"2024-04-23T18:20:30.807355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665585+0000","last_change":"2024-04-23T18:19:42.003042+0000","last_active":"2024-04-23T18:20:30.665585+0000","last_peered":"2024-04-23T18:20:30.665585+0000","last_clean":"2024-04-23T18:20:30.665585+0000","last_became_active":"2024-04-23T18:19:42.002728+0000","last_became_peered":"2024-04-23T18:19:42.002728+0000","last_unstale":"2024-04-23T18:20:30.665585+0000","last_undegraded":"2024-04-23T18:20:30.665585+0000","last_fullsized":"2024-04-23T18:20:30.665585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:54:33.401918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662782+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:20:30.662782+0000","last_peered":"2024-04-23T18:20:30.662782+0000","last_clean":"2024-04-23T18:20:30.662782+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:20:30.662782+0000","last_undegraded":"2024-04-23T18:20:30.662782+0000","last_fullsized":"2024-04-23T18:20:30.662782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808223+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:20:30.808223+0000","last_peered":"2024-04-23T18:20:30.808223+0000","last_clean":"2024-04-23T18:20:30.808223+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:20:30.808223+0000","last_undegraded":"2024-04-23T18:20:30.808223+0000","last_fullsized":"2024-04-23T18:20:30.808223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656274+0000","last_change":"2024-04-23T18:19:41.691129+0000","last_active":"2024-04-23T18:20:30.656274+0000","last_peered":"2024-04-23T18:20:30.656274+0000","last_clean":"2024-04-23T18:20:30.656274+0000","last_became_active":"2024-04-23T18:19:41.690710+0000","last_became_peered":"2024-04-23T18:19:41.690710+0000","last_unstale":"2024-04-23T18:20:30.656274+0000","last_undegraded":"2024-04-23T18:20:30.656274+0000","last_fullsized":"2024-04-23T18:20:30.656274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:51.914794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559388+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:20:30.559388+0000","last_peered":"2024-04-23T18:20:30.559388+0000","last_clean":"2024-04-23T18:20:30.559388+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:20:30.559388+0000","last_undegraded":"2024-04-23T18:20:30.559388+0000","last_fullsized":"2024-04-23T18:20:30.559388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806758+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:20:30.806758+0000","last_peered":"2024-04-23T18:20:30.806758+0000","last_clean":"2024-04-23T18:20:30.806758+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:20:30.806758+0000","last_undegraded":"2024-04-23T18:20:30.806758+0000","last_fullsized":"2024-04-23T18:20:30.806758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806380+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:20:30.806380+0000","last_peered":"2024-04-23T18:20:30.806380+0000","last_clean":"2024-04-23T18:20:30.806380+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:20:30.806380+0000","last_undegraded":"2024-04-23T18:20:30.806380+0000","last_fullsized":"2024-04-23T18:20:30.806380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806975+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:20:30.806975+0000","last_peered":"2024-04-23T18:20:30.806975+0000","last_clean":"2024-04-23T18:20:30.806975+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:20:30.806975+0000","last_undegraded":"2024-04-23T18:20:30.806975+0000","last_fullsized":"2024-04-23T18:20:30.806975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655969+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:20:30.655969+0000","last_peered":"2024-04-23T18:20:30.655969+0000","last_clean":"2024-04-23T18:20:30.655969+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:20:30.655969+0000","last_undegraded":"2024-04-23T18:20:30.655969+0000","last_fullsized":"2024-04-23T18:20:30.655969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806739+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:20:30.806739+0000","last_peered":"2024-04-23T18:20:30.806739+0000","last_clean":"2024-04-23T18:20:30.806739+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:20:30.806739+0000","last_undegraded":"2024-04-23T18:20:30.806739+0000","last_fullsized":"2024-04-23T18:20:30.806739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'45","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.774689+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:20:30.774689+0000","last_peered":"2024-04-23T18:20:30.774689+0000","last_clean":"2024-04-23T18:20:30.774689+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:20:30.774689+0000","last_undegraded":"2024-04-23T18:20:30.774689+0000","last_fullsized":"2024-04-23T18:20:30.774689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656706+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:20:30.656706+0000","last_peered":"2024-04-23T18:20:30.656706+0000","last_clean":"2024-04-23T18:20:30.656706+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:20:30.656706+0000","last_undegraded":"2024-04-23T18:20:30.656706+0000","last_fullsized":"2024-04-23T18:20:30.656706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559381+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:20:30.559381+0000","last_peered":"2024-04-23T18:20:30.559381+0000","last_clean":"2024-04-23T18:20:30.559381+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:20:30.559381+0000","last_undegraded":"2024-04-23T18:20:30.559381+0000","last_fullsized":"2024-04-23T18:20:30.559381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809178+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:20:30.809178+0000","last_peered":"2024-04-23T18:20:30.809178+0000","last_clean":"2024-04-23T18:20:30.809178+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:20:30.809178+0000","last_undegraded":"2024-04-23T18:20:30.809178+0000","last_fullsized":"2024-04-23T18:20:30.809178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806198+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:20:30.806198+0000","last_peered":"2024-04-23T18:20:30.806198+0000","last_clean":"2024-04-23T18:20:30.806198+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:20:30.806198+0000","last_undegraded":"2024-04-23T18:20:30.806198+0000","last_fullsized":"2024-04-23T18:20:30.806198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655622+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:20:30.655622+0000","last_peered":"2024-04-23T18:20:30.655622+0000","last_clean":"2024-04-23T18:20:30.655622+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:20:30.655622+0000","last_undegraded":"2024-04-23T18:20:30.655622+0000","last_fullsized":"2024-04-23T18:20:30.655622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655976+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:20:30.655976+0000","last_peered":"2024-04-23T18:20:30.655976+0000","last_clean":"2024-04-23T18:20:30.655976+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:20:30.655976+0000","last_undegraded":"2024-04-23T18:20:30.655976+0000","last_fullsized":"2024-04-23T18:20:30.655976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109391+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:20:31.109391+0000","last_peered":"2024-04-23T18:20:31.109391+0000","last_clean":"2024-04-23T18:20:31.109391+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:20:31.109391+0000","last_undegraded":"2024-04-23T18:20:31.109391+0000","last_fullsized":"2024-04-23T18:20:31.109391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562843+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:20:30.562843+0000","last_peered":"2024-04-23T18:20:30.562843+0000","last_clean":"2024-04-23T18:20:30.562843+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:20:30.562843+0000","last_undegraded":"2024-04-23T18:20:30.562843+0000","last_fullsized":"2024-04-23T18:20:30.562843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655954+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:20:30.655954+0000","last_peered":"2024-04-23T18:20:30.655954+0000","last_clean":"2024-04-23T18:20:30.655954+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:20:30.655954+0000","last_undegraded":"2024-04-23T18:20:30.655954+0000","last_fullsized":"2024-04-23T18:20:30.655954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655726+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:20:30.655726+0000","last_peered":"2024-04-23T18:20:30.655726+0000","last_clean":"2024-04-23T18:20:30.655726+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:20:30.655726+0000","last_undegraded":"2024-04-23T18:20:30.655726+0000","last_fullsized":"2024-04-23T18:20:30.655726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807464+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:20:30.807464+0000","last_peered":"2024-04-23T18:20:30.807464+0000","last_clean":"2024-04-23T18:20:30.807464+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:20:30.807464+0000","last_undegraded":"2024-04-23T18:20:30.807464+0000","last_fullsized":"2024-04-23T18:20:30.807464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562850+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:20:30.562850+0000","last_peered":"2024-04-23T18:20:30.562850+0000","last_clean":"2024-04-23T18:20:30.562850+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:20:30.562850+0000","last_undegraded":"2024-04-23T18:20:30.562850+0000","last_fullsized":"2024-04-23T18:20:30.562850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807721+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:20:30.807721+0000","last_peered":"2024-04-23T18:20:30.807721+0000","last_clean":"2024-04-23T18:20:30.807721+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:20:30.807721+0000","last_undegraded":"2024-04-23T18:20:30.807721+0000","last_fullsized":"2024-04-23T18:20:30.807721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.770167+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:20:30.770167+0000","last_peered":"2024-04-23T18:20:30.770167+0000","last_clean":"2024-04-23T18:20:30.770167+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:20:30.770167+0000","last_undegraded":"2024-04-23T18:20:30.770167+0000","last_fullsized":"2024-04-23T18:20:30.770167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807610+0000","last_change":"2024-04-23T18:19:41.698372+0000","last_active":"2024-04-23T18:20:30.807610+0000","last_peered":"2024-04-23T18:20:30.807610+0000","last_clean":"2024-04-23T18:20:30.807610+0000","last_became_active":"2024-04-23T18:19:41.695014+0000","last_became_peered":"2024-04-23T18:19:41.695014+0000","last_unstale":"2024-04-23T18:20:30.807610+0000","last_undegraded":"2024-04-23T18:20:30.807610+0000","last_fullsized":"2024-04-23T18:20:30.807610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:35:15.568241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806730+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:20:30.806730+0000","last_peered":"2024-04-23T18:20:30.806730+0000","last_clean":"2024-04-23T18:20:30.806730+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:20:30.806730+0000","last_undegraded":"2024-04-23T18:20:30.806730+0000","last_fullsized":"2024-04-23T18:20:30.806730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807869+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:20:30.807869+0000","last_peered":"2024-04-23T18:20:30.807869+0000","last_clean":"2024-04-23T18:20:30.807869+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:20:30.807869+0000","last_undegraded":"2024-04-23T18:20:30.807869+0000","last_fullsized":"2024-04-23T18:20:30.807869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109472+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:20:31.109472+0000","last_peered":"2024-04-23T18:20:31.109472+0000","last_clean":"2024-04-23T18:20:31.109472+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:20:31.109472+0000","last_undegraded":"2024-04-23T18:20:31.109472+0000","last_fullsized":"2024-04-23T18:20:31.109472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562560+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:20:30.562560+0000","last_peered":"2024-04-23T18:20:30.562560+0000","last_clean":"2024-04-23T18:20:30.562560+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:20:30.562560+0000","last_undegraded":"2024-04-23T18:20:30.562560+0000","last_fullsized":"2024-04-23T18:20:30.562560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562670+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:20:30.562670+0000","last_peered":"2024-04-23T18:20:30.562670+0000","last_clean":"2024-04-23T18:20:30.562670+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:20:30.562670+0000","last_undegraded":"2024-04-23T18:20:30.562670+0000","last_fullsized":"2024-04-23T18:20:30.562670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808727+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:20:30.808727+0000","last_peered":"2024-04-23T18:20:30.808727+0000","last_clean":"2024-04-23T18:20:30.808727+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:20:30.808727+0000","last_undegraded":"2024-04-23T18:20:30.808727+0000","last_fullsized":"2024-04-23T18:20:30.808727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806849+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:20:30.806849+0000","last_peered":"2024-04-23T18:20:30.806849+0000","last_clean":"2024-04-23T18:20:30.806849+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:20:30.806849+0000","last_undegraded":"2024-04-23T18:20:30.806849+0000","last_fullsized":"2024-04-23T18:20:30.806849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109541+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:20:31.109541+0000","last_peered":"2024-04-23T18:20:31.109541+0000","last_clean":"2024-04-23T18:20:31.109541+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:20:31.109541+0000","last_undegraded":"2024-04-23T18:20:31.109541+0000","last_fullsized":"2024-04-23T18:20:31.109541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562953+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:20:30.562953+0000","last_peered":"2024-04-23T18:20:30.562953+0000","last_clean":"2024-04-23T18:20:30.562953+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:20:30.562953+0000","last_undegraded":"2024-04-23T18:20:30.562953+0000","last_fullsized":"2024-04-23T18:20:30.562953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656683+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:20:30.656683+0000","last_peered":"2024-04-23T18:20:30.656683+0000","last_clean":"2024-04-23T18:20:30.656683+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:20:30.656683+0000","last_undegraded":"2024-04-23T18:20:30.656683+0000","last_fullsized":"2024-04-23T18:20:30.656683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662620+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:20:30.662620+0000","last_peered":"2024-04-23T18:20:30.662620+0000","last_clean":"2024-04-23T18:20:30.662620+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:20:30.662620+0000","last_undegraded":"2024-04-23T18:20:30.662620+0000","last_fullsized":"2024-04-23T18:20:30.662620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655927+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:20:30.655927+0000","last_peered":"2024-04-23T18:20:30.655927+0000","last_clean":"2024-04-23T18:20:30.655927+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:20:30.655927+0000","last_undegraded":"2024-04-23T18:20:30.655927+0000","last_fullsized":"2024-04-23T18:20:30.655927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806424+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:20:30.806424+0000","last_peered":"2024-04-23T18:20:30.806424+0000","last_clean":"2024-04-23T18:20:30.806424+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:20:30.806424+0000","last_undegraded":"2024-04-23T18:20:30.806424+0000","last_fullsized":"2024-04-23T18:20:30.806424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806659+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:20:30.806659+0000","last_peered":"2024-04-23T18:20:30.806659+0000","last_clean":"2024-04-23T18:20:30.806659+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:20:30.806659+0000","last_undegraded":"2024-04-23T18:20:30.806659+0000","last_fullsized":"2024-04-23T18:20:30.806659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109535+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:20:31.109535+0000","last_peered":"2024-04-23T18:20:31.109535+0000","last_clean":"2024-04-23T18:20:31.109535+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:20:31.109535+0000","last_undegraded":"2024-04-23T18:20:31.109535+0000","last_fullsized":"2024-04-23T18:20:31.109535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597639,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":790,"num_read_kb":505,"num_write":285,"num_write_kb":1436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":61080,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1792917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285603,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":472865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1712128,"data_stored":1141296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1417216,"data_stored":875364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005107,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1417216,"data_stored":875284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1613824,"data_stored":1074945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":481420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":548971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":482691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1006430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":550538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37232,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707920,"statfs":{"total":95995035648,"available":95956910080,"internally_reserved":0,"allocated":1757184,"data_stored":1205189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:20:36.315 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-23T18:20:36.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:20:36.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 -- 172.21.15.37:0/214373326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c101a50 msgr2=0x7fd19c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/214373326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c101a50 0x7fd19c105a90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd184009900 tx=0x7fd18402a410 comp rx=0 tx=0).stop 2024-04-23T18:20:36.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 -- 172.21.15.37:0/214373326 shutdown_connections 2024-04-23T18:20:36.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/214373326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c106170 0x7fd19c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/214373326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c101a50 0x7fd19c105a90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/214373326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c1010a0 0x7fd19c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 -- 172.21.15.37:0/214373326 >> 172.21.15.37:0/214373326 conn(0x7fd19c077f50 msgr2=0x7fd19c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 -- 172.21.15.37:0/214373326 shutdown_connections 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.460+0000 7fd1a31d8640 1 -- 172.21.15.37:0/214373326 wait complete. 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 Processor -- start 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 -- start start 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c1010a0 0x7fd19c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c106170 0x7fd19c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 0x7fd19c1a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c111090 con 0x7fd19c19a000 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c110f10 con 0x7fd19c106170 2024-04-23T18:20:36.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a31d8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c111210 con 0x7fd19c1010a0 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 0x7fd19c1a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 0x7fd19c1a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43028/0 (socket says 172.21.15.37:43028) 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 -- 172.21.15.37:0/610367009 learned_addr learned my addr 172.21.15.37:0/610367009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a0f74640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c1010a0 0x7fd19c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd19bfff640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c106170 0x7fd19c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 -- 172.21.15.37:0/610367009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c1010a0 msgr2=0x7fd19c199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c1010a0 0x7fd19c199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 -- 172.21.15.37:0/610367009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c106170 msgr2=0x7fd19c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c106170 0x7fd19c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 -- 172.21.15.37:0/610367009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c1a4db0 con 0x7fd19c19a000 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd19bfff640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c106170 0x7fd19c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.461+0000 7fd1a1775640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 0x7fd19c1a45b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd19400cce0 tx=0x7fd19400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:36.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.462+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19400f450 con 0x7fd19c19a000 2024-04-23T18:20:36.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.462+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19400faa0 con 0x7fd19c19a000 2024-04-23T18:20:36.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.462+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c1a50a0 con 0x7fd19c19a000 2024-04-23T18:20:36.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.462+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194018b70 con 0x7fd19c19a000 2024-04-23T18:20:36.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.462+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1a55c0 con 0x7fd19c19a000 2024-04-23T18:20:36.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.463+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c068d70 con 0x7fd19c19a000 2024-04-23T18:20:36.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.464+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fd194018d10 con 0x7fd19c19a000 2024-04-23T18:20:36.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.464+0000 7fd199ffb640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd1880a3e30 0x7fd1880a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.465+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fd1940d0d00 con 0x7fd19c19a000 2024-04-23T18:20:36.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.465+0000 7fd1a0f74640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd1880a3e30 0x7fd1880a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.465+0000 7fd1a0f74640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd1880a3e30 0x7fd1880a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00bbe0 tx=0x7fd18c00b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:36.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.469+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd194014030 con 0x7fd19c19a000 2024-04-23T18:20:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.655+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd19c101a50 con 0x7fd1880a3e30 2024-04-23T18:20:36.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.662+0000 7fd199ffb640 1 -- 172.21.15.37:0/610367009 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257765 (secure 0 0 0) 0x7fd19c101a50 con 0x7fd1880a3e30 2024-04-23T18:20:36.659 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:36.661 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-23T18:20:36.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.665+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd1880a3e30 msgr2=0x7fd1880a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.665+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd1880a3e30 0x7fd1880a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00bbe0 tx=0x7fd18c00b390 comp rx=0 tx=0).stop 2024-04-23T18:20:36.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.665+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 msgr2=0x7fd19c1a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.665+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 0x7fd19c1a45b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd19400cce0 tx=0x7fd19400a620 comp rx=0 tx=0).stop 2024-04-23T18:20:36.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 shutdown_connections 2024-04-23T18:20:36.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd1880a3e30 0x7fd1880a62f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c19a000 0x7fd19c1a45b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd19c106170 0x7fd19c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 --2- 172.21.15.37:0/610367009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd19c1010a0 0x7fd19c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 >> 172.21.15.37:0/610367009 conn(0x7fd19c077f50 msgr2=0x7fd19c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:36.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 shutdown_connections 2024-04-23T18:20:36.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.666+0000 7fd1a31d8640 1 -- 172.21.15.37:0/610367009 wait complete. 2024-04-23T18:20:36.671 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T18:20:36.572107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188241,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1156,"num_read_kb":831,"num_write":567,"num_write_kb":4824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1241933,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":917,"ondisk_log_size":917,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389636,"kb_used_data":15312,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552188,"statfs":{"total":1151940427776,"available":1151541440512,"internally_reserved":0,"allocated":15679488,"data_stored":9198054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":230,"num_read_kb":207,"num_write":99,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990339"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109725+0000","last_change":"2024-04-23T18:18:35.059712+0000","last_active":"2024-04-23T18:20:31.109725+0000","last_peered":"2024-04-23T18:20:31.109725+0000","last_clean":"2024-04-23T18:20:31.109725+0000","last_became_active":"2024-04-23T18:18:35.058871+0000","last_became_peered":"2024-04-23T18:18:35.058871+0000","last_unstale":"2024-04-23T18:20:31.109725+0000","last_undegraded":"2024-04-23T18:20:31.109725+0000","last_fullsized":"2024-04-23T18:20:31.109725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:26:49.644191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809044+0000","last_change":"2024-04-23T18:18:35.066934+0000","last_active":"2024-04-23T18:20:30.809044+0000","last_peered":"2024-04-23T18:20:30.809044+0000","last_clean":"2024-04-23T18:20:30.809044+0000","last_became_active":"2024-04-23T18:18:35.066565+0000","last_became_peered":"2024-04-23T18:18:35.066565+0000","last_unstale":"2024-04-23T18:20:30.809044+0000","last_undegraded":"2024-04-23T18:20:30.809044+0000","last_fullsized":"2024-04-23T18:20:30.809044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:29.560614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808851+0000","last_change":"2024-04-23T18:18:35.058911+0000","last_active":"2024-04-23T18:20:30.808851+0000","last_peered":"2024-04-23T18:20:30.808851+0000","last_clean":"2024-04-23T18:20:30.808851+0000","last_became_active":"2024-04-23T18:18:35.058527+0000","last_became_peered":"2024-04-23T18:18:35.058527+0000","last_unstale":"2024-04-23T18:20:30.808851+0000","last_undegraded":"2024-04-23T18:20:30.808851+0000","last_fullsized":"2024-04-23T18:20:30.808851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:40.333077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557782+0000","last_change":"2024-04-23T18:19:41.694639+0000","last_active":"2024-04-23T18:20:30.557782+0000","last_peered":"2024-04-23T18:20:30.557782+0000","last_clean":"2024-04-23T18:20:30.557782+0000","last_became_active":"2024-04-23T18:19:41.694291+0000","last_became_peered":"2024-04-23T18:19:41.694291+0000","last_unstale":"2024-04-23T18:20:30.557782+0000","last_undegraded":"2024-04-23T18:20:30.557782+0000","last_fullsized":"2024-04-23T18:20:30.557782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:18:44.268737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655753+0000","last_change":"2024-04-23T18:18:35.061017+0000","last_active":"2024-04-23T18:20:30.655753+0000","last_peered":"2024-04-23T18:20:30.655753+0000","last_clean":"2024-04-23T18:20:30.655753+0000","last_became_active":"2024-04-23T18:18:35.060671+0000","last_became_peered":"2024-04-23T18:18:35.060671+0000","last_unstale":"2024-04-23T18:20:30.655753+0000","last_undegraded":"2024-04-23T18:20:30.655753+0000","last_fullsized":"2024-04-23T18:20:30.655753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:33.356275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665534+0000","last_change":"2024-04-23T18:18:35.067772+0000","last_active":"2024-04-23T18:20:30.665534+0000","last_peered":"2024-04-23T18:20:30.665534+0000","last_clean":"2024-04-23T18:20:30.665534+0000","last_became_active":"2024-04-23T18:18:35.067316+0000","last_became_peered":"2024-04-23T18:18:35.067316+0000","last_unstale":"2024-04-23T18:20:30.665534+0000","last_undegraded":"2024-04-23T18:20:30.665534+0000","last_fullsized":"2024-04-23T18:20:30.665534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:58.992491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806744+0000","last_change":"2024-04-23T18:18:35.058381+0000","last_active":"2024-04-23T18:20:30.806744+0000","last_peered":"2024-04-23T18:20:30.806744+0000","last_clean":"2024-04-23T18:20:30.806744+0000","last_became_active":"2024-04-23T18:18:35.058030+0000","last_became_peered":"2024-04-23T18:18:35.058030+0000","last_unstale":"2024-04-23T18:20:30.806744+0000","last_undegraded":"2024-04-23T18:20:30.806744+0000","last_fullsized":"2024-04-23T18:20:30.806744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:24:40.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655576+0000","last_change":"2024-04-23T18:18:35.068486+0000","last_active":"2024-04-23T18:20:30.655576+0000","last_peered":"2024-04-23T18:20:30.655576+0000","last_clean":"2024-04-23T18:20:30.655576+0000","last_became_active":"2024-04-23T18:18:35.067661+0000","last_became_peered":"2024-04-23T18:18:35.067661+0000","last_unstale":"2024-04-23T18:20:30.655576+0000","last_undegraded":"2024-04-23T18:20:30.655576+0000","last_fullsized":"2024-04-23T18:20:30.655576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:17:15.526977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807041+0000","last_change":"2024-04-23T18:18:35.059876+0000","last_active":"2024-04-23T18:20:30.807041+0000","last_peered":"2024-04-23T18:20:30.807041+0000","last_clean":"2024-04-23T18:20:30.807041+0000","last_became_active":"2024-04-23T18:18:35.059507+0000","last_became_peered":"2024-04-23T18:18:35.059507+0000","last_unstale":"2024-04-23T18:20:30.807041+0000","last_undegraded":"2024-04-23T18:20:30.807041+0000","last_fullsized":"2024-04-23T18:20:30.807041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:45:35.583674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665845+0000","last_change":"2024-04-23T18:18:35.070658+0000","last_active":"2024-04-23T18:20:30.665845+0000","last_peered":"2024-04-23T18:20:30.665845+0000","last_clean":"2024-04-23T18:20:30.665845+0000","last_became_active":"2024-04-23T18:18:35.070336+0000","last_became_peered":"2024-04-23T18:18:35.070336+0000","last_unstale":"2024-04-23T18:20:30.665845+0000","last_undegraded":"2024-04-23T18:20:30.665845+0000","last_fullsized":"2024-04-23T18:20:30.665845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:10:50.617459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807817+0000","last_change":"2024-04-23T18:18:35.047765+0000","last_active":"2024-04-23T18:20:30.807817+0000","last_peered":"2024-04-23T18:20:30.807817+0000","last_clean":"2024-04-23T18:20:30.807817+0000","last_became_active":"2024-04-23T18:18:35.047409+0000","last_became_peered":"2024-04-23T18:18:35.047409+0000","last_unstale":"2024-04-23T18:20:30.807817+0000","last_undegraded":"2024-04-23T18:20:30.807817+0000","last_fullsized":"2024-04-23T18:20:30.807817+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:53:25.122267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559597+0000","last_change":"2024-04-23T18:18:35.072564+0000","last_active":"2024-04-23T18:20:30.559597+0000","last_peered":"2024-04-23T18:20:30.559597+0000","last_clean":"2024-04-23T18:20:30.559597+0000","last_became_active":"2024-04-23T18:18:35.072142+0000","last_became_peered":"2024-04-23T18:18:35.072142+0000","last_unstale":"2024-04-23T18:20:30.559597+0000","last_undegraded":"2024-04-23T18:20:30.559597+0000","last_fullsized":"2024-04-23T18:20:30.559597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:39.104749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806683+0000","last_change":"2024-04-23T18:18:35.053579+0000","last_active":"2024-04-23T18:20:30.806683+0000","last_peered":"2024-04-23T18:20:30.806683+0000","last_clean":"2024-04-23T18:20:30.806683+0000","last_became_active":"2024-04-23T18:18:35.052274+0000","last_became_peered":"2024-04-23T18:18:35.052274+0000","last_unstale":"2024-04-23T18:20:30.806683+0000","last_undegraded":"2024-04-23T18:20:30.806683+0000","last_fullsized":"2024-04-23T18:20:30.806683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:36:50.760357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808485+0000","last_change":"2024-04-23T18:18:35.065893+0000","last_active":"2024-04-23T18:20:30.808485+0000","last_peered":"2024-04-23T18:20:30.808485+0000","last_clean":"2024-04-23T18:20:30.808485+0000","last_became_active":"2024-04-23T18:18:35.065506+0000","last_became_peered":"2024-04-23T18:18:35.065506+0000","last_unstale":"2024-04-23T18:20:30.808485+0000","last_undegraded":"2024-04-23T18:20:30.808485+0000","last_fullsized":"2024-04-23T18:20:30.808485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:06.481186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808365+0000","last_change":"2024-04-23T18:18:35.059713+0000","last_active":"2024-04-23T18:20:30.808365+0000","last_peered":"2024-04-23T18:20:30.808365+0000","last_clean":"2024-04-23T18:20:30.808365+0000","last_became_active":"2024-04-23T18:18:35.059336+0000","last_became_peered":"2024-04-23T18:18:35.059336+0000","last_unstale":"2024-04-23T18:20:30.808365+0000","last_undegraded":"2024-04-23T18:20:30.808365+0000","last_fullsized":"2024-04-23T18:20:30.808365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:04:25.846836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.805909+0000","last_change":"2024-04-23T18:18:35.069439+0000","last_active":"2024-04-23T18:20:30.805909+0000","last_peered":"2024-04-23T18:20:30.805909+0000","last_clean":"2024-04-23T18:20:30.805909+0000","last_became_active":"2024-04-23T18:18:35.068948+0000","last_became_peered":"2024-04-23T18:18:35.068948+0000","last_unstale":"2024-04-23T18:20:30.805909+0000","last_undegraded":"2024-04-23T18:20:30.805909+0000","last_fullsized":"2024-04-23T18:20:30.805909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:44.541120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109804+0000","last_change":"2024-04-23T18:18:35.060040+0000","last_active":"2024-04-23T18:20:31.109804+0000","last_peered":"2024-04-23T18:20:31.109804+0000","last_clean":"2024-04-23T18:20:31.109804+0000","last_became_active":"2024-04-23T18:18:35.059356+0000","last_became_peered":"2024-04-23T18:18:35.059356+0000","last_unstale":"2024-04-23T18:20:31.109804+0000","last_undegraded":"2024-04-23T18:20:31.109804+0000","last_fullsized":"2024-04-23T18:20:31.109804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:14.500286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557930+0000","last_change":"2024-04-23T18:18:35.064296+0000","last_active":"2024-04-23T18:20:30.557930+0000","last_peered":"2024-04-23T18:20:30.557930+0000","last_clean":"2024-04-23T18:20:30.557930+0000","last_became_active":"2024-04-23T18:18:35.063965+0000","last_became_peered":"2024-04-23T18:18:35.063965+0000","last_unstale":"2024-04-23T18:20:30.557930+0000","last_undegraded":"2024-04-23T18:20:30.557930+0000","last_fullsized":"2024-04-23T18:20:30.557930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:03.089317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562690+0000","last_change":"2024-04-23T18:18:35.068934+0000","last_active":"2024-04-23T18:20:30.562690+0000","last_peered":"2024-04-23T18:20:30.562690+0000","last_clean":"2024-04-23T18:20:30.562690+0000","last_became_active":"2024-04-23T18:18:35.068508+0000","last_became_peered":"2024-04-23T18:18:35.068508+0000","last_unstale":"2024-04-23T18:20:30.562690+0000","last_undegraded":"2024-04-23T18:20:30.562690+0000","last_fullsized":"2024-04-23T18:20:30.562690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:44:31.069162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.558026+0000","last_change":"2024-04-23T18:18:34.051547+0000","last_active":"2024-04-23T18:20:30.558026+0000","last_peered":"2024-04-23T18:20:30.558026+0000","last_clean":"2024-04-23T18:20:30.558026+0000","last_became_active":"2024-04-23T18:18:34.051135+0000","last_became_peered":"2024-04-23T18:18:34.051135+0000","last_unstale":"2024-04-23T18:20:30.558026+0000","last_undegraded":"2024-04-23T18:20:30.558026+0000","last_fullsized":"2024-04-23T18:20:30.558026+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:05:41.832499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656540+0000","last_change":"2024-04-23T18:18:35.069629+0000","last_active":"2024-04-23T18:20:30.656540+0000","last_peered":"2024-04-23T18:20:30.656540+0000","last_clean":"2024-04-23T18:20:30.656540+0000","last_became_active":"2024-04-23T18:18:35.069278+0000","last_became_peered":"2024-04-23T18:18:35.069278+0000","last_unstale":"2024-04-23T18:20:30.656540+0000","last_undegraded":"2024-04-23T18:20:30.656540+0000","last_fullsized":"2024-04-23T18:20:30.656540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:14:30.133741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656813+0000","last_change":"2024-04-23T18:18:34.055045+0000","last_active":"2024-04-23T18:20:30.656813+0000","last_peered":"2024-04-23T18:20:30.656813+0000","last_clean":"2024-04-23T18:20:30.656813+0000","last_became_active":"2024-04-23T18:18:34.054387+0000","last_became_peered":"2024-04-23T18:18:34.054387+0000","last_unstale":"2024-04-23T18:20:30.656813+0000","last_undegraded":"2024-04-23T18:20:30.656813+0000","last_fullsized":"2024-04-23T18:20:30.656813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:53.144990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809178+0000","last_change":"2024-04-23T18:18:35.057249+0000","last_active":"2024-04-23T18:20:30.809178+0000","last_peered":"2024-04-23T18:20:30.809178+0000","last_clean":"2024-04-23T18:20:30.809178+0000","last_became_active":"2024-04-23T18:18:35.056911+0000","last_became_peered":"2024-04-23T18:18:35.056911+0000","last_unstale":"2024-04-23T18:20:30.809178+0000","last_undegraded":"2024-04-23T18:20:30.809178+0000","last_fullsized":"2024-04-23T18:20:30.809178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:31.331726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656091+0000","last_change":"2024-04-23T18:18:34.048055+0000","last_active":"2024-04-23T18:20:30.656091+0000","last_peered":"2024-04-23T18:20:30.656091+0000","last_clean":"2024-04-23T18:20:30.656091+0000","last_became_active":"2024-04-23T18:18:34.047643+0000","last_became_peered":"2024-04-23T18:18:34.047643+0000","last_unstale":"2024-04-23T18:20:30.656091+0000","last_undegraded":"2024-04-23T18:20:30.656091+0000","last_fullsized":"2024-04-23T18:20:30.656091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:44:08.078276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806401+0000","last_change":"2024-04-23T18:18:35.064259+0000","last_active":"2024-04-23T18:20:30.806401+0000","last_peered":"2024-04-23T18:20:30.806401+0000","last_clean":"2024-04-23T18:20:30.806401+0000","last_became_active":"2024-04-23T18:18:35.063911+0000","last_became_peered":"2024-04-23T18:18:35.063911+0000","last_unstale":"2024-04-23T18:20:30.806401+0000","last_undegraded":"2024-04-23T18:20:30.806401+0000","last_fullsized":"2024-04-23T18:20:30.806401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:08:56.078241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655978+0000","last_change":"2024-04-23T18:18:34.047590+0000","last_active":"2024-04-23T18:20:30.655978+0000","last_peered":"2024-04-23T18:20:30.655978+0000","last_clean":"2024-04-23T18:20:30.655978+0000","last_became_active":"2024-04-23T18:18:34.047191+0000","last_became_peered":"2024-04-23T18:18:34.047191+0000","last_unstale":"2024-04-23T18:20:30.655978+0000","last_undegraded":"2024-04-23T18:20:30.655978+0000","last_fullsized":"2024-04-23T18:20:30.655978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:39.377102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662948+0000","last_change":"2024-04-23T18:18:35.059553+0000","last_active":"2024-04-23T18:20:30.662948+0000","last_peered":"2024-04-23T18:20:30.662948+0000","last_clean":"2024-04-23T18:20:30.662948+0000","last_became_active":"2024-04-23T18:18:35.059200+0000","last_became_peered":"2024-04-23T18:18:35.059200+0000","last_unstale":"2024-04-23T18:20:30.662948+0000","last_undegraded":"2024-04-23T18:20:30.662948+0000","last_fullsized":"2024-04-23T18:20:30.662948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:09.392729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":144,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662757+0000","last_change":"2024-04-23T18:18:34.050051+0000","last_active":"2024-04-23T18:20:30.662757+0000","last_peered":"2024-04-23T18:20:30.662757+0000","last_clean":"2024-04-23T18:20:30.662757+0000","last_became_active":"2024-04-23T18:18:34.049478+0000","last_became_peered":"2024-04-23T18:18:34.049478+0000","last_unstale":"2024-04-23T18:20:30.662757+0000","last_undegraded":"2024-04-23T18:20:30.662757+0000","last_fullsized":"2024-04-23T18:20:30.662757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:04.211197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655725+0000","last_change":"2024-04-23T18:18:35.055519+0000","last_active":"2024-04-23T18:20:30.655725+0000","last_peered":"2024-04-23T18:20:30.655725+0000","last_clean":"2024-04-23T18:20:30.655725+0000","last_became_active":"2024-04-23T18:18:35.054159+0000","last_became_peered":"2024-04-23T18:18:35.054159+0000","last_unstale":"2024-04-23T18:20:30.655725+0000","last_undegraded":"2024-04-23T18:20:30.655725+0000","last_fullsized":"2024-04-23T18:20:30.655725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:20:06.072240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665477+0000","last_change":"2024-04-23T18:19:41.933594+0000","last_active":"2024-04-23T18:20:30.665477+0000","last_peered":"2024-04-23T18:20:30.665477+0000","last_clean":"2024-04-23T18:20:30.665477+0000","last_became_active":"2024-04-23T18:19:41.930226+0000","last_became_peered":"2024-04-23T18:19:41.930226+0000","last_unstale":"2024-04-23T18:20:30.665477+0000","last_undegraded":"2024-04-23T18:20:30.665477+0000","last_fullsized":"2024-04-23T18:20:30.665477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:54:57.183719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662758+0000","last_change":"2024-04-23T18:19:41.694345+0000","last_active":"2024-04-23T18:20:30.662758+0000","last_peered":"2024-04-23T18:20:30.662758+0000","last_clean":"2024-04-23T18:20:30.662758+0000","last_became_active":"2024-04-23T18:19:41.693912+0000","last_became_peered":"2024-04-23T18:19:41.693912+0000","last_unstale":"2024-04-23T18:20:30.662758+0000","last_undegraded":"2024-04-23T18:20:30.662758+0000","last_fullsized":"2024-04-23T18:20:30.662758+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:12:07.153403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806576+0000","last_change":"2024-04-23T18:18:34.055192+0000","last_active":"2024-04-23T18:20:30.806576+0000","last_peered":"2024-04-23T18:20:30.806576+0000","last_clean":"2024-04-23T18:20:30.806576+0000","last_became_active":"2024-04-23T18:18:34.054881+0000","last_became_peered":"2024-04-23T18:18:34.054881+0000","last_unstale":"2024-04-23T18:20:30.806576+0000","last_undegraded":"2024-04-23T18:20:30.806576+0000","last_fullsized":"2024-04-23T18:20:30.806576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:42:19.534358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655788+0000","last_change":"2024-04-23T18:18:35.056564+0000","last_active":"2024-04-23T18:20:30.655788+0000","last_peered":"2024-04-23T18:20:30.655788+0000","last_clean":"2024-04-23T18:20:30.655788+0000","last_became_active":"2024-04-23T18:18:35.056234+0000","last_became_peered":"2024-04-23T18:18:35.056234+0000","last_unstale":"2024-04-23T18:20:30.655788+0000","last_undegraded":"2024-04-23T18:20:30.655788+0000","last_fullsized":"2024-04-23T18:20:30.655788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:28.297019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662554+0000","last_change":"2024-04-23T18:18:34.049913+0000","last_active":"2024-04-23T18:20:30.662554+0000","last_peered":"2024-04-23T18:20:30.662554+0000","last_clean":"2024-04-23T18:20:30.662554+0000","last_became_active":"2024-04-23T18:18:34.049552+0000","last_became_peered":"2024-04-23T18:18:34.049552+0000","last_unstale":"2024-04-23T18:20:30.662554+0000","last_undegraded":"2024-04-23T18:20:30.662554+0000","last_fullsized":"2024-04-23T18:20:30.662554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:33.455730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557710+0000","last_change":"2024-04-23T18:18:35.057248+0000","last_active":"2024-04-23T18:20:30.557710+0000","last_peered":"2024-04-23T18:20:30.557710+0000","last_clean":"2024-04-23T18:20:30.557710+0000","last_became_active":"2024-04-23T18:18:35.056924+0000","last_became_peered":"2024-04-23T18:18:35.056924+0000","last_unstale":"2024-04-23T18:20:30.557710+0000","last_undegraded":"2024-04-23T18:20:30.557710+0000","last_fullsized":"2024-04-23T18:20:30.557710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:22:48.594757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":137,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665821+0000","last_change":"2024-04-23T18:18:34.052252+0000","last_active":"2024-04-23T18:20:30.665821+0000","last_peered":"2024-04-23T18:20:30.665821+0000","last_clean":"2024-04-23T18:20:30.665821+0000","last_became_active":"2024-04-23T18:18:34.051923+0000","last_became_peered":"2024-04-23T18:18:34.051923+0000","last_unstale":"2024-04-23T18:20:30.665821+0000","last_undegraded":"2024-04-23T18:20:30.665821+0000","last_fullsized":"2024-04-23T18:20:30.665821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:16:18.824365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806707+0000","last_change":"2024-04-23T18:18:35.063916+0000","last_active":"2024-04-23T18:20:30.806707+0000","last_peered":"2024-04-23T18:20:30.806707+0000","last_clean":"2024-04-23T18:20:30.806707+0000","last_became_active":"2024-04-23T18:18:35.063543+0000","last_became_peered":"2024-04-23T18:18:35.063543+0000","last_unstale":"2024-04-23T18:20:30.806707+0000","last_undegraded":"2024-04-23T18:20:30.806707+0000","last_fullsized":"2024-04-23T18:20:30.806707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:13:02.469253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807724+0000","last_change":"2024-04-23T18:18:34.048404+0000","last_active":"2024-04-23T18:20:30.807724+0000","last_peered":"2024-04-23T18:20:30.807724+0000","last_clean":"2024-04-23T18:20:30.807724+0000","last_became_active":"2024-04-23T18:18:34.048054+0000","last_became_peered":"2024-04-23T18:18:34.048054+0000","last_unstale":"2024-04-23T18:20:30.807724+0000","last_undegraded":"2024-04-23T18:20:30.807724+0000","last_fullsized":"2024-04-23T18:20:30.807724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:26:43.946699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809126+0000","last_change":"2024-04-23T18:18:35.060448+0000","last_active":"2024-04-23T18:20:30.809126+0000","last_peered":"2024-04-23T18:20:30.809126+0000","last_clean":"2024-04-23T18:20:30.809126+0000","last_became_active":"2024-04-23T18:18:35.060088+0000","last_became_peered":"2024-04-23T18:18:35.060088+0000","last_unstale":"2024-04-23T18:20:30.809126+0000","last_undegraded":"2024-04-23T18:20:30.809126+0000","last_fullsized":"2024-04-23T18:20:30.809126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:23:48.581665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662978+0000","last_change":"2024-04-23T18:18:34.053467+0000","last_active":"2024-04-23T18:20:30.662978+0000","last_peered":"2024-04-23T18:20:30.662978+0000","last_clean":"2024-04-23T18:20:30.662978+0000","last_became_active":"2024-04-23T18:18:34.052960+0000","last_became_peered":"2024-04-23T18:18:34.052960+0000","last_unstale":"2024-04-23T18:20:30.662978+0000","last_undegraded":"2024-04-23T18:20:30.662978+0000","last_fullsized":"2024-04-23T18:20:30.662978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:18:41.861701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806650+0000","last_change":"2024-04-23T18:19:41.694590+0000","last_active":"2024-04-23T18:20:30.806650+0000","last_peered":"2024-04-23T18:20:30.806650+0000","last_clean":"2024-04-23T18:20:30.806650+0000","last_became_active":"2024-04-23T18:19:41.694163+0000","last_became_peered":"2024-04-23T18:19:41.694163+0000","last_unstale":"2024-04-23T18:20:30.806650+0000","last_undegraded":"2024-04-23T18:20:30.806650+0000","last_fullsized":"2024-04-23T18:20:30.806650+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:35:28.978205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":308,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.007812+0000","last_change":"2024-04-23T18:19:41.940140+0000","last_active":"2024-04-23T18:20:31.007812+0000","last_peered":"2024-04-23T18:20:31.007812+0000","last_clean":"2024-04-23T18:20:31.007812+0000","last_became_active":"2024-04-23T18:19:41.931221+0000","last_became_peered":"2024-04-23T18:19:41.931221+0000","last_unstale":"2024-04-23T18:20:31.007812+0000","last_undegraded":"2024-04-23T18:20:31.007812+0000","last_fullsized":"2024-04-23T18:20:31.007812+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:40.122956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":227,"num_read_kb":179,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61046,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655901+0000","last_change":"2024-04-23T18:18:35.046804+0000","last_active":"2024-04-23T18:20:30.655901+0000","last_peered":"2024-04-23T18:20:30.655901+0000","last_clean":"2024-04-23T18:20:30.655901+0000","last_became_active":"2024-04-23T18:18:35.046447+0000","last_became_peered":"2024-04-23T18:18:35.046447+0000","last_unstale":"2024-04-23T18:20:30.655901+0000","last_undegraded":"2024-04-23T18:20:30.655901+0000","last_fullsized":"2024-04-23T18:20:30.655901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:10:45.733721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109715+0000","last_change":"2024-04-23T18:19:41.699880+0000","last_active":"2024-04-23T18:20:31.109715+0000","last_peered":"2024-04-23T18:20:31.109715+0000","last_clean":"2024-04-23T18:20:31.109715+0000","last_became_active":"2024-04-23T18:19:41.691650+0000","last_became_peered":"2024-04-23T18:19:41.691650+0000","last_unstale":"2024-04-23T18:20:31.109715+0000","last_undegraded":"2024-04-23T18:20:31.109715+0000","last_fullsized":"2024-04-23T18:20:31.109715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:10:28.166844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806532+0000","last_change":"2024-04-23T18:18:35.065860+0000","last_active":"2024-04-23T18:20:30.806532+0000","last_peered":"2024-04-23T18:20:30.806532+0000","last_clean":"2024-04-23T18:20:30.806532+0000","last_became_active":"2024-04-23T18:18:35.065323+0000","last_became_peered":"2024-04-23T18:18:35.065323+0000","last_unstale":"2024-04-23T18:20:30.806532+0000","last_undegraded":"2024-04-23T18:20:30.806532+0000","last_fullsized":"2024-04-23T18:20:30.806532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:02:08.097006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662756+0000","last_change":"2024-04-23T18:18:34.052586+0000","last_active":"2024-04-23T18:20:30.662756+0000","last_peered":"2024-04-23T18:20:30.662756+0000","last_clean":"2024-04-23T18:20:30.662756+0000","last_became_active":"2024-04-23T18:18:34.049537+0000","last_became_peered":"2024-04-23T18:18:34.049537+0000","last_unstale":"2024-04-23T18:20:30.662756+0000","last_undegraded":"2024-04-23T18:20:30.662756+0000","last_fullsized":"2024-04-23T18:20:30.662756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:50.464965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807217+0000","last_change":"2024-04-23T18:18:35.062904+0000","last_active":"2024-04-23T18:20:30.807217+0000","last_peered":"2024-04-23T18:20:30.807217+0000","last_clean":"2024-04-23T18:20:30.807217+0000","last_became_active":"2024-04-23T18:18:35.062481+0000","last_became_peered":"2024-04-23T18:18:35.062481+0000","last_unstale":"2024-04-23T18:20:30.807217+0000","last_undegraded":"2024-04-23T18:20:30.807217+0000","last_fullsized":"2024-04-23T18:20:30.807217+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:59.454972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":144,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.260785+0000","last_change":"2024-04-23T18:18:34.049190+0000","last_active":"2024-04-23T18:20:31.260785+0000","last_peered":"2024-04-23T18:20:31.260785+0000","last_clean":"2024-04-23T18:20:31.260785+0000","last_became_active":"2024-04-23T18:18:34.047760+0000","last_became_peered":"2024-04-23T18:18:34.047760+0000","last_unstale":"2024-04-23T18:20:31.260785+0000","last_undegraded":"2024-04-23T18:20:31.260785+0000","last_fullsized":"2024-04-23T18:20:31.260785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:23.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806508+0000","last_change":"2024-04-23T18:18:35.052109+0000","last_active":"2024-04-23T18:20:30.806508+0000","last_peered":"2024-04-23T18:20:30.806508+0000","last_clean":"2024-04-23T18:20:30.806508+0000","last_became_active":"2024-04-23T18:18:35.051661+0000","last_became_peered":"2024-04-23T18:18:35.051661+0000","last_unstale":"2024-04-23T18:20:30.806508+0000","last_undegraded":"2024-04-23T18:20:30.806508+0000","last_fullsized":"2024-04-23T18:20:30.806508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:34:18.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655631+0000","last_change":"2024-04-23T18:18:34.051793+0000","last_active":"2024-04-23T18:20:30.655631+0000","last_peered":"2024-04-23T18:20:30.655631+0000","last_clean":"2024-04-23T18:20:30.655631+0000","last_became_active":"2024-04-23T18:18:34.051483+0000","last_became_peered":"2024-04-23T18:18:34.051483+0000","last_unstale":"2024-04-23T18:20:30.655631+0000","last_undegraded":"2024-04-23T18:20:30.655631+0000","last_fullsized":"2024-04-23T18:20:30.655631+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:19:59.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662693+0000","last_change":"2024-04-23T18:18:35.058412+0000","last_active":"2024-04-23T18:20:30.662693+0000","last_peered":"2024-04-23T18:20:30.662693+0000","last_clean":"2024-04-23T18:20:30.662693+0000","last_became_active":"2024-04-23T18:18:35.057738+0000","last_became_peered":"2024-04-23T18:18:35.057738+0000","last_unstale":"2024-04-23T18:20:30.662693+0000","last_undegraded":"2024-04-23T18:20:30.662693+0000","last_fullsized":"2024-04-23T18:20:30.662693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:18.876572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656007+0000","last_change":"2024-04-23T18:18:34.050346+0000","last_active":"2024-04-23T18:20:30.656007+0000","last_peered":"2024-04-23T18:20:30.656007+0000","last_clean":"2024-04-23T18:20:30.656007+0000","last_became_active":"2024-04-23T18:18:34.049939+0000","last_became_peered":"2024-04-23T18:18:34.049939+0000","last_unstale":"2024-04-23T18:20:30.656007+0000","last_undegraded":"2024-04-23T18:20:30.656007+0000","last_fullsized":"2024-04-23T18:20:30.656007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:49:11.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109660+0000","last_change":"2024-04-23T18:18:35.065457+0000","last_active":"2024-04-23T18:20:31.109660+0000","last_peered":"2024-04-23T18:20:31.109660+0000","last_clean":"2024-04-23T18:20:31.109660+0000","last_became_active":"2024-04-23T18:18:35.065116+0000","last_became_peered":"2024-04-23T18:18:35.065116+0000","last_unstale":"2024-04-23T18:20:31.109660+0000","last_undegraded":"2024-04-23T18:20:31.109660+0000","last_fullsized":"2024-04-23T18:20:31.109660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:36:44.569728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.557746+0000","last_change":"2024-04-23T18:18:34.053504+0000","last_active":"2024-04-23T18:20:30.557746+0000","last_peered":"2024-04-23T18:20:30.557746+0000","last_clean":"2024-04-23T18:20:30.557746+0000","last_became_active":"2024-04-23T18:18:34.053122+0000","last_became_peered":"2024-04-23T18:18:34.053122+0000","last_unstale":"2024-04-23T18:20:30.557746+0000","last_undegraded":"2024-04-23T18:20:30.557746+0000","last_fullsized":"2024-04-23T18:20:30.557746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:59:15.987470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665629+0000","last_change":"2024-04-23T18:18:35.062631+0000","last_active":"2024-04-23T18:20:30.665629+0000","last_peered":"2024-04-23T18:20:30.665629+0000","last_clean":"2024-04-23T18:20:30.665629+0000","last_became_active":"2024-04-23T18:18:35.062285+0000","last_became_peered":"2024-04-23T18:18:35.062285+0000","last_unstale":"2024-04-23T18:20:30.665629+0000","last_undegraded":"2024-04-23T18:20:30.665629+0000","last_fullsized":"2024-04-23T18:20:30.665629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:45:34.258939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807355+0000","last_change":"2024-04-23T18:18:34.052848+0000","last_active":"2024-04-23T18:20:30.807355+0000","last_peered":"2024-04-23T18:20:30.807355+0000","last_clean":"2024-04-23T18:20:30.807355+0000","last_became_active":"2024-04-23T18:18:34.052337+0000","last_became_peered":"2024-04-23T18:18:34.052337+0000","last_unstale":"2024-04-23T18:20:30.807355+0000","last_undegraded":"2024-04-23T18:20:30.807355+0000","last_fullsized":"2024-04-23T18:20:30.807355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:41:08.276642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.665585+0000","last_change":"2024-04-23T18:19:42.003042+0000","last_active":"2024-04-23T18:20:30.665585+0000","last_peered":"2024-04-23T18:20:30.665585+0000","last_clean":"2024-04-23T18:20:30.665585+0000","last_became_active":"2024-04-23T18:19:42.002728+0000","last_became_peered":"2024-04-23T18:19:42.002728+0000","last_unstale":"2024-04-23T18:20:30.665585+0000","last_undegraded":"2024-04-23T18:20:30.665585+0000","last_fullsized":"2024-04-23T18:20:30.665585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:54:33.401918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662782+0000","last_change":"2024-04-23T18:18:35.065904+0000","last_active":"2024-04-23T18:20:30.662782+0000","last_peered":"2024-04-23T18:20:30.662782+0000","last_clean":"2024-04-23T18:20:30.662782+0000","last_became_active":"2024-04-23T18:18:35.065597+0000","last_became_peered":"2024-04-23T18:18:35.065597+0000","last_unstale":"2024-04-23T18:20:30.662782+0000","last_undegraded":"2024-04-23T18:20:30.662782+0000","last_fullsized":"2024-04-23T18:20:30.662782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:02.514228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808223+0000","last_change":"2024-04-23T18:18:34.054354+0000","last_active":"2024-04-23T18:20:30.808223+0000","last_peered":"2024-04-23T18:20:30.808223+0000","last_clean":"2024-04-23T18:20:30.808223+0000","last_became_active":"2024-04-23T18:18:34.053368+0000","last_became_peered":"2024-04-23T18:18:34.053368+0000","last_unstale":"2024-04-23T18:20:30.808223+0000","last_undegraded":"2024-04-23T18:20:30.808223+0000","last_fullsized":"2024-04-23T18:20:30.808223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:06:05.109473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656274+0000","last_change":"2024-04-23T18:19:41.691129+0000","last_active":"2024-04-23T18:20:30.656274+0000","last_peered":"2024-04-23T18:20:30.656274+0000","last_clean":"2024-04-23T18:20:30.656274+0000","last_became_active":"2024-04-23T18:19:41.690710+0000","last_became_peered":"2024-04-23T18:19:41.690710+0000","last_unstale":"2024-04-23T18:20:30.656274+0000","last_undegraded":"2024-04-23T18:20:30.656274+0000","last_fullsized":"2024-04-23T18:20:30.656274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:21:51.914794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559388+0000","last_change":"2024-04-23T18:18:34.048733+0000","last_active":"2024-04-23T18:20:30.559388+0000","last_peered":"2024-04-23T18:20:30.559388+0000","last_clean":"2024-04-23T18:20:30.559388+0000","last_became_active":"2024-04-23T18:18:34.047709+0000","last_became_peered":"2024-04-23T18:18:34.047709+0000","last_unstale":"2024-04-23T18:20:30.559388+0000","last_undegraded":"2024-04-23T18:20:30.559388+0000","last_fullsized":"2024-04-23T18:20:30.559388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:05:15.544415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806758+0000","last_change":"2024-04-23T18:17:27.724008+0000","last_active":"2024-04-23T18:20:30.806758+0000","last_peered":"2024-04-23T18:20:30.806758+0000","last_clean":"2024-04-23T18:20:30.806758+0000","last_became_active":"2024-04-23T18:17:27.723476+0000","last_became_peered":"2024-04-23T18:17:27.723476+0000","last_unstale":"2024-04-23T18:20:30.806758+0000","last_undegraded":"2024-04-23T18:20:30.806758+0000","last_fullsized":"2024-04-23T18:20:30.806758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:14:06.094841+0000","last_clean_scrub_stamp":"2024-04-23T18:14:06.094841+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:13:35.955088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806380+0000","last_change":"2024-04-23T18:18:35.071901+0000","last_active":"2024-04-23T18:20:30.806380+0000","last_peered":"2024-04-23T18:20:30.806380+0000","last_clean":"2024-04-23T18:20:30.806380+0000","last_became_active":"2024-04-23T18:18:35.071558+0000","last_became_peered":"2024-04-23T18:18:35.071558+0000","last_unstale":"2024-04-23T18:20:30.806380+0000","last_undegraded":"2024-04-23T18:20:30.806380+0000","last_fullsized":"2024-04-23T18:20:30.806380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:45:32.332875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806975+0000","last_change":"2024-04-23T18:18:34.050767+0000","last_active":"2024-04-23T18:20:30.806975+0000","last_peered":"2024-04-23T18:20:30.806975+0000","last_clean":"2024-04-23T18:20:30.806975+0000","last_became_active":"2024-04-23T18:18:34.050426+0000","last_became_peered":"2024-04-23T18:18:34.050426+0000","last_unstale":"2024-04-23T18:20:30.806975+0000","last_undegraded":"2024-04-23T18:20:30.806975+0000","last_fullsized":"2024-04-23T18:20:30.806975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:41:26.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655969+0000","last_change":"2024-04-23T18:18:34.057940+0000","last_active":"2024-04-23T18:20:30.655969+0000","last_peered":"2024-04-23T18:20:30.655969+0000","last_clean":"2024-04-23T18:20:30.655969+0000","last_became_active":"2024-04-23T18:18:34.057588+0000","last_became_peered":"2024-04-23T18:18:34.057588+0000","last_unstale":"2024-04-23T18:20:30.655969+0000","last_undegraded":"2024-04-23T18:20:30.655969+0000","last_fullsized":"2024-04-23T18:20:30.655969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:15.750253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806739+0000","last_change":"2024-04-23T18:18:35.062507+0000","last_active":"2024-04-23T18:20:30.806739+0000","last_peered":"2024-04-23T18:20:30.806739+0000","last_clean":"2024-04-23T18:20:30.806739+0000","last_became_active":"2024-04-23T18:18:35.062152+0000","last_became_peered":"2024-04-23T18:18:35.062152+0000","last_unstale":"2024-04-23T18:20:30.806739+0000","last_undegraded":"2024-04-23T18:20:30.806739+0000","last_fullsized":"2024-04-23T18:20:30.806739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:33.423659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'98","reported_seq":211,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:32.798177+0000","last_change":"2024-04-23T18:18:34.051913+0000","last_active":"2024-04-23T18:20:32.798177+0000","last_peered":"2024-04-23T18:20:32.798177+0000","last_clean":"2024-04-23T18:20:32.798177+0000","last_became_active":"2024-04-23T18:18:34.051492+0000","last_became_peered":"2024-04-23T18:18:34.051492+0000","last_unstale":"2024-04-23T18:20:32.798177+0000","last_undegraded":"2024-04-23T18:20:32.798177+0000","last_fullsized":"2024-04-23T18:20:32.798177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:47:25.346754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":179,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656706+0000","last_change":"2024-04-23T18:18:35.060672+0000","last_active":"2024-04-23T18:20:30.656706+0000","last_peered":"2024-04-23T18:20:30.656706+0000","last_clean":"2024-04-23T18:20:30.656706+0000","last_became_active":"2024-04-23T18:18:35.059607+0000","last_became_peered":"2024-04-23T18:18:35.059607+0000","last_unstale":"2024-04-23T18:20:30.656706+0000","last_undegraded":"2024-04-23T18:20:30.656706+0000","last_fullsized":"2024-04-23T18:20:30.656706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:24.765618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.559381+0000","last_change":"2024-04-23T18:18:34.058414+0000","last_active":"2024-04-23T18:20:30.559381+0000","last_peered":"2024-04-23T18:20:30.559381+0000","last_clean":"2024-04-23T18:20:30.559381+0000","last_became_active":"2024-04-23T18:18:34.058040+0000","last_became_peered":"2024-04-23T18:18:34.058040+0000","last_unstale":"2024-04-23T18:20:30.559381+0000","last_undegraded":"2024-04-23T18:20:30.559381+0000","last_fullsized":"2024-04-23T18:20:30.559381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:09.990916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.809178+0000","last_change":"2024-04-23T18:18:35.061560+0000","last_active":"2024-04-23T18:20:30.809178+0000","last_peered":"2024-04-23T18:20:30.809178+0000","last_clean":"2024-04-23T18:20:30.809178+0000","last_became_active":"2024-04-23T18:18:35.060812+0000","last_became_peered":"2024-04-23T18:18:35.060812+0000","last_unstale":"2024-04-23T18:20:30.809178+0000","last_undegraded":"2024-04-23T18:20:30.809178+0000","last_fullsized":"2024-04-23T18:20:30.809178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:24:28.431797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806198+0000","last_change":"2024-04-23T18:18:34.057375+0000","last_active":"2024-04-23T18:20:30.806198+0000","last_peered":"2024-04-23T18:20:30.806198+0000","last_clean":"2024-04-23T18:20:30.806198+0000","last_became_active":"2024-04-23T18:18:34.056975+0000","last_became_peered":"2024-04-23T18:18:34.056975+0000","last_unstale":"2024-04-23T18:20:30.806198+0000","last_undegraded":"2024-04-23T18:20:30.806198+0000","last_fullsized":"2024-04-23T18:20:30.806198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:19:02.380630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655622+0000","last_change":"2024-04-23T18:18:35.050519+0000","last_active":"2024-04-23T18:20:30.655622+0000","last_peered":"2024-04-23T18:20:30.655622+0000","last_clean":"2024-04-23T18:20:30.655622+0000","last_became_active":"2024-04-23T18:18:35.050185+0000","last_became_peered":"2024-04-23T18:18:35.050185+0000","last_unstale":"2024-04-23T18:20:30.655622+0000","last_undegraded":"2024-04-23T18:20:30.655622+0000","last_fullsized":"2024-04-23T18:20:30.655622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:27:23.771525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655976+0000","last_change":"2024-04-23T18:18:34.054402+0000","last_active":"2024-04-23T18:20:30.655976+0000","last_peered":"2024-04-23T18:20:30.655976+0000","last_clean":"2024-04-23T18:20:30.655976+0000","last_became_active":"2024-04-23T18:18:34.054003+0000","last_became_peered":"2024-04-23T18:18:34.054003+0000","last_unstale":"2024-04-23T18:20:30.655976+0000","last_undegraded":"2024-04-23T18:20:30.655976+0000","last_fullsized":"2024-04-23T18:20:30.655976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:03:17.176642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109391+0000","last_change":"2024-04-23T18:18:35.065683+0000","last_active":"2024-04-23T18:20:31.109391+0000","last_peered":"2024-04-23T18:20:31.109391+0000","last_clean":"2024-04-23T18:20:31.109391+0000","last_became_active":"2024-04-23T18:18:35.063323+0000","last_became_peered":"2024-04-23T18:18:35.063323+0000","last_unstale":"2024-04-23T18:20:31.109391+0000","last_undegraded":"2024-04-23T18:20:31.109391+0000","last_fullsized":"2024-04-23T18:20:31.109391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:48:29.344956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562843+0000","last_change":"2024-04-23T18:18:34.058683+0000","last_active":"2024-04-23T18:20:30.562843+0000","last_peered":"2024-04-23T18:20:30.562843+0000","last_clean":"2024-04-23T18:20:30.562843+0000","last_became_active":"2024-04-23T18:18:34.058290+0000","last_became_peered":"2024-04-23T18:18:34.058290+0000","last_unstale":"2024-04-23T18:20:30.562843+0000","last_undegraded":"2024-04-23T18:20:30.562843+0000","last_fullsized":"2024-04-23T18:20:30.562843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:16:09.836861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655954+0000","last_change":"2024-04-23T18:18:35.067372+0000","last_active":"2024-04-23T18:20:30.655954+0000","last_peered":"2024-04-23T18:20:30.655954+0000","last_clean":"2024-04-23T18:20:30.655954+0000","last_became_active":"2024-04-23T18:18:35.066774+0000","last_became_peered":"2024-04-23T18:18:35.066774+0000","last_unstale":"2024-04-23T18:20:30.655954+0000","last_undegraded":"2024-04-23T18:20:30.655954+0000","last_fullsized":"2024-04-23T18:20:30.655954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:24:02.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655726+0000","last_change":"2024-04-23T18:18:34.049019+0000","last_active":"2024-04-23T18:20:30.655726+0000","last_peered":"2024-04-23T18:20:30.655726+0000","last_clean":"2024-04-23T18:20:30.655726+0000","last_became_active":"2024-04-23T18:18:34.048655+0000","last_became_peered":"2024-04-23T18:18:34.048655+0000","last_unstale":"2024-04-23T18:20:30.655726+0000","last_undegraded":"2024-04-23T18:20:30.655726+0000","last_fullsized":"2024-04-23T18:20:30.655726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:01:29.967069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807464+0000","last_change":"2024-04-23T18:18:35.062935+0000","last_active":"2024-04-23T18:20:30.807464+0000","last_peered":"2024-04-23T18:20:30.807464+0000","last_clean":"2024-04-23T18:20:30.807464+0000","last_became_active":"2024-04-23T18:18:35.062537+0000","last_became_peered":"2024-04-23T18:18:35.062537+0000","last_unstale":"2024-04-23T18:20:30.807464+0000","last_undegraded":"2024-04-23T18:20:30.807464+0000","last_fullsized":"2024-04-23T18:20:30.807464+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:22:09.402249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562850+0000","last_change":"2024-04-23T18:18:34.058425+0000","last_active":"2024-04-23T18:20:30.562850+0000","last_peered":"2024-04-23T18:20:30.562850+0000","last_clean":"2024-04-23T18:20:30.562850+0000","last_became_active":"2024-04-23T18:18:34.058042+0000","last_became_peered":"2024-04-23T18:18:34.058042+0000","last_unstale":"2024-04-23T18:20:30.562850+0000","last_undegraded":"2024-04-23T18:20:30.562850+0000","last_fullsized":"2024-04-23T18:20:30.562850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:03:11.123330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807721+0000","last_change":"2024-04-23T18:18:35.047825+0000","last_active":"2024-04-23T18:20:30.807721+0000","last_peered":"2024-04-23T18:20:30.807721+0000","last_clean":"2024-04-23T18:20:30.807721+0000","last_became_active":"2024-04-23T18:18:35.047486+0000","last_became_peered":"2024-04-23T18:18:35.047486+0000","last_unstale":"2024-04-23T18:20:30.807721+0000","last_undegraded":"2024-04-23T18:20:30.807721+0000","last_fullsized":"2024-04-23T18:20:30.807721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:57:02.187138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.770167+0000","last_change":"2024-04-23T18:18:34.048269+0000","last_active":"2024-04-23T18:20:30.770167+0000","last_peered":"2024-04-23T18:20:30.770167+0000","last_clean":"2024-04-23T18:20:30.770167+0000","last_became_active":"2024-04-23T18:18:34.047917+0000","last_became_peered":"2024-04-23T18:18:34.047917+0000","last_unstale":"2024-04-23T18:20:30.770167+0000","last_undegraded":"2024-04-23T18:20:30.770167+0000","last_fullsized":"2024-04-23T18:20:30.770167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:19:23.565493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807610+0000","last_change":"2024-04-23T18:19:41.698372+0000","last_active":"2024-04-23T18:20:30.807610+0000","last_peered":"2024-04-23T18:20:30.807610+0000","last_clean":"2024-04-23T18:20:30.807610+0000","last_became_active":"2024-04-23T18:19:41.695014+0000","last_became_peered":"2024-04-23T18:19:41.695014+0000","last_unstale":"2024-04-23T18:20:30.807610+0000","last_undegraded":"2024-04-23T18:20:30.807610+0000","last_fullsized":"2024-04-23T18:20:30.807610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:35:15.568241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806730+0000","last_change":"2024-04-23T18:18:34.049749+0000","last_active":"2024-04-23T18:20:30.806730+0000","last_peered":"2024-04-23T18:20:30.806730+0000","last_clean":"2024-04-23T18:20:30.806730+0000","last_became_active":"2024-04-23T18:18:34.049367+0000","last_became_peered":"2024-04-23T18:18:34.049367+0000","last_unstale":"2024-04-23T18:20:30.806730+0000","last_undegraded":"2024-04-23T18:20:30.806730+0000","last_fullsized":"2024-04-23T18:20:30.806730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:33.019683+0000","last_clean_scrub_stamp":"2024-04-23T18:18:33.019683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:29:40.051197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.807869+0000","last_change":"2024-04-23T18:18:35.062194+0000","last_active":"2024-04-23T18:20:30.807869+0000","last_peered":"2024-04-23T18:20:30.807869+0000","last_clean":"2024-04-23T18:20:30.807869+0000","last_became_active":"2024-04-23T18:18:35.061706+0000","last_became_peered":"2024-04-23T18:18:35.061706+0000","last_unstale":"2024-04-23T18:20:30.807869+0000","last_undegraded":"2024-04-23T18:20:30.807869+0000","last_fullsized":"2024-04-23T18:20:30.807869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:49.576946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109472+0000","last_change":"2024-04-23T18:18:35.066194+0000","last_active":"2024-04-23T18:20:31.109472+0000","last_peered":"2024-04-23T18:20:31.109472+0000","last_clean":"2024-04-23T18:20:31.109472+0000","last_became_active":"2024-04-23T18:18:35.065750+0000","last_became_peered":"2024-04-23T18:18:35.065750+0000","last_unstale":"2024-04-23T18:20:31.109472+0000","last_undegraded":"2024-04-23T18:20:31.109472+0000","last_fullsized":"2024-04-23T18:20:31.109472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:18:50.140203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562560+0000","last_change":"2024-04-23T18:18:35.069268+0000","last_active":"2024-04-23T18:20:30.562560+0000","last_peered":"2024-04-23T18:20:30.562560+0000","last_clean":"2024-04-23T18:20:30.562560+0000","last_became_active":"2024-04-23T18:18:35.065099+0000","last_became_peered":"2024-04-23T18:18:35.065099+0000","last_unstale":"2024-04-23T18:20:30.562560+0000","last_undegraded":"2024-04-23T18:20:30.562560+0000","last_fullsized":"2024-04-23T18:20:30.562560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:03:10.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562670+0000","last_change":"2024-04-23T18:18:35.067250+0000","last_active":"2024-04-23T18:20:30.562670+0000","last_peered":"2024-04-23T18:20:30.562670+0000","last_clean":"2024-04-23T18:20:30.562670+0000","last_became_active":"2024-04-23T18:18:35.064175+0000","last_became_peered":"2024-04-23T18:18:35.064175+0000","last_unstale":"2024-04-23T18:20:30.562670+0000","last_undegraded":"2024-04-23T18:20:30.562670+0000","last_fullsized":"2024-04-23T18:20:30.562670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:36:33.784237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.808727+0000","last_change":"2024-04-23T18:18:35.060393+0000","last_active":"2024-04-23T18:20:30.808727+0000","last_peered":"2024-04-23T18:20:30.808727+0000","last_clean":"2024-04-23T18:20:30.808727+0000","last_became_active":"2024-04-23T18:18:35.060028+0000","last_became_peered":"2024-04-23T18:18:35.060028+0000","last_unstale":"2024-04-23T18:20:30.808727+0000","last_undegraded":"2024-04-23T18:20:30.808727+0000","last_fullsized":"2024-04-23T18:20:30.808727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:35:14.470709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806849+0000","last_change":"2024-04-23T18:18:35.061814+0000","last_active":"2024-04-23T18:20:30.806849+0000","last_peered":"2024-04-23T18:20:30.806849+0000","last_clean":"2024-04-23T18:20:30.806849+0000","last_became_active":"2024-04-23T18:18:35.061459+0000","last_became_peered":"2024-04-23T18:18:35.061459+0000","last_unstale":"2024-04-23T18:20:30.806849+0000","last_undegraded":"2024-04-23T18:20:30.806849+0000","last_fullsized":"2024-04-23T18:20:30.806849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:21:24.439279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109541+0000","last_change":"2024-04-23T18:18:35.061104+0000","last_active":"2024-04-23T18:20:31.109541+0000","last_peered":"2024-04-23T18:20:31.109541+0000","last_clean":"2024-04-23T18:20:31.109541+0000","last_became_active":"2024-04-23T18:18:35.060758+0000","last_became_peered":"2024-04-23T18:18:35.060758+0000","last_unstale":"2024-04-23T18:20:31.109541+0000","last_undegraded":"2024-04-23T18:20:31.109541+0000","last_fullsized":"2024-04-23T18:20:31.109541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:31:36.145262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.562953+0000","last_change":"2024-04-23T18:18:35.058103+0000","last_active":"2024-04-23T18:20:30.562953+0000","last_peered":"2024-04-23T18:20:30.562953+0000","last_clean":"2024-04-23T18:20:30.562953+0000","last_became_active":"2024-04-23T18:18:35.057366+0000","last_became_peered":"2024-04-23T18:18:35.057366+0000","last_unstale":"2024-04-23T18:20:30.562953+0000","last_undegraded":"2024-04-23T18:20:30.562953+0000","last_fullsized":"2024-04-23T18:20:30.562953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:10:12.277995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.656683+0000","last_change":"2024-04-23T18:18:35.071389+0000","last_active":"2024-04-23T18:20:30.656683+0000","last_peered":"2024-04-23T18:20:30.656683+0000","last_clean":"2024-04-23T18:20:30.656683+0000","last_became_active":"2024-04-23T18:18:35.070303+0000","last_became_peered":"2024-04-23T18:18:35.070303+0000","last_unstale":"2024-04-23T18:20:30.656683+0000","last_undegraded":"2024-04-23T18:20:30.656683+0000","last_fullsized":"2024-04-23T18:20:30.656683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:06.354517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.662620+0000","last_change":"2024-04-23T18:18:35.062570+0000","last_active":"2024-04-23T18:20:30.662620+0000","last_peered":"2024-04-23T18:20:30.662620+0000","last_clean":"2024-04-23T18:20:30.662620+0000","last_became_active":"2024-04-23T18:18:35.062138+0000","last_became_peered":"2024-04-23T18:18:35.062138+0000","last_unstale":"2024-04-23T18:20:30.662620+0000","last_undegraded":"2024-04-23T18:20:30.662620+0000","last_fullsized":"2024-04-23T18:20:30.662620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:30:15.542710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.655927+0000","last_change":"2024-04-23T18:18:35.073225+0000","last_active":"2024-04-23T18:20:30.655927+0000","last_peered":"2024-04-23T18:20:30.655927+0000","last_clean":"2024-04-23T18:20:30.655927+0000","last_became_active":"2024-04-23T18:18:35.072841+0000","last_became_peered":"2024-04-23T18:18:35.072841+0000","last_unstale":"2024-04-23T18:20:30.655927+0000","last_undegraded":"2024-04-23T18:20:30.655927+0000","last_fullsized":"2024-04-23T18:20:30.655927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:17.134003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806424+0000","last_change":"2024-04-23T18:18:35.067317+0000","last_active":"2024-04-23T18:20:30.806424+0000","last_peered":"2024-04-23T18:20:30.806424+0000","last_clean":"2024-04-23T18:20:30.806424+0000","last_became_active":"2024-04-23T18:18:35.066849+0000","last_became_peered":"2024-04-23T18:18:35.066849+0000","last_unstale":"2024-04-23T18:20:30.806424+0000","last_undegraded":"2024-04-23T18:20:30.806424+0000","last_fullsized":"2024-04-23T18:20:30.806424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:03.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:30.806659+0000","last_change":"2024-04-23T18:18:35.064805+0000","last_active":"2024-04-23T18:20:30.806659+0000","last_peered":"2024-04-23T18:20:30.806659+0000","last_clean":"2024-04-23T18:20:30.806659+0000","last_became_active":"2024-04-23T18:18:35.064358+0000","last_became_peered":"2024-04-23T18:18:35.064358+0000","last_unstale":"2024-04-23T18:20:30.806659+0000","last_undegraded":"2024-04-23T18:20:30.806659+0000","last_fullsized":"2024-04-23T18:20:30.806659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:20:26.538764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-23T18:20:31.109535+0000","last_change":"2024-04-23T18:18:35.061722+0000","last_active":"2024-04-23T18:20:31.109535+0000","last_peered":"2024-04-23T18:20:31.109535+0000","last_clean":"2024-04-23T18:20:31.109535+0000","last_became_active":"2024-04-23T18:18:35.061402+0000","last_became_peered":"2024-04-23T18:18:35.061402+0000","last_unstale":"2024-04-23T18:20:31.109535+0000","last_undegraded":"2024-04-23T18:20:31.109535+0000","last_fullsized":"2024-04-23T18:20:31.109535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:18:34.025579+0000","last_clean_scrub_stamp":"2024-04-23T18:18:34.025579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:39:15.954961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597639,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1020,"num_read_kb":712,"num_write":384,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":61080,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1792917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285603,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":472865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547241,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1712128,"data_stored":1141296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743471,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36720,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708432,"statfs":{"total":95995035648,"available":95957434368,"internally_reserved":0,"allocated":1417216,"data_stored":875364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005107,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1417216,"data_stored":875284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1613824,"data_stored":1074945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":481420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":548971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":482691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":483061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1006430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":550538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37232,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707920,"statfs":{"total":95995035648,"available":95956910080,"internally_reserved":0,"allocated":1757184,"data_stored":1205189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:20:36.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 -- 172.21.15.37:0/2787098964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8314107280 msgr2=0x7f831410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 --2- 172.21.15.37:0/2787098964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8314107280 0x7f831410ee50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f830c009ca0 tx=0x7f830c02a8e0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 -- 172.21.15.37:0/2787098964 shutdown_connections 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 --2- 172.21.15.37:0/2787098964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8314107280 0x7f831410ee50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 --2- 172.21.15.37:0/2787098964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8314102bf0 0x7f8314106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 --2- 172.21.15.37:0/2787098964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8314102240 0x7f8314102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 -- 172.21.15.37:0/2787098964 >> 172.21.15.37:0/2787098964 conn(0x7f83140790e0 msgr2=0x7f83140794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 -- 172.21.15.37:0/2787098964 shutdown_connections 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 -- 172.21.15.37:0/2787098964 wait complete. 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 Processor -- start 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.819+0000 7f831a564640 1 -- start start 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f831a564640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8314102240 0x7f8314198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f831a564640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8314107280 0x7f8314198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f831a564640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 0x7f831419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f831a564640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8314112240 con 0x7f831419cfe0 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f831a564640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83141120c0 con 0x7f8314107280 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f831a564640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83141123c0 con 0x7f8314102240 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 0x7f831419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 0x7f831419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43050/0 (socket says 172.21.15.37:43050) 2024-04-23T18:20:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 -- 172.21.15.37:0/271893114 learned_addr learned my addr 172.21.15.37:0/271893114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8313fff640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8314102240 0x7f8314198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f83137fe640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8314107280 0x7f8314198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 -- 172.21.15.37:0/271893114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8314102240 msgr2=0x7f8314198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8314102240 0x7f8314198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 -- 172.21.15.37:0/271893114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8314107280 msgr2=0x7f8314198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8314107280 0x7f8314198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 -- 172.21.15.37:0/271893114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831419dad0 con 0x7f831419cfe0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.820+0000 7f8318b01640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 0x7f831419d490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f830c005cc0 tx=0x7f830c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.821+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c004040 con 0x7f831419cfe0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.821+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831419dd60 con 0x7f831419cfe0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.821+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c004690 con 0x7f831419cfe0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.821+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c030e70 con 0x7f831419cfe0 2024-04-23T18:20:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.821+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83141a5560 con 0x7f831419cfe0 2024-04-23T18:20:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.822+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83140691c0 con 0x7f831419cfe0 2024-04-23T18:20:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.824+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f830c0305e0 con 0x7f831419cfe0 2024-04-23T18:20:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.824+0000 7f83117fa640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f82fc0a3db0 0x7f82fc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.824+0000 7f8313fff640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f82fc0a3db0 0x7f82fc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.824+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f830c0f1f70 con 0x7f831419cfe0 2024-04-23T18:20:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.825+0000 7f8313fff640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f82fc0a3db0 0x7f82fc0a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83040058c0 tx=0x7f830400a4c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:36.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:36.828+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f830c0be5c0 con 0x7f831419cfe0 2024-04-23T18:20:37.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.056+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8314102bf0 con 0x7f831419cfe0 2024-04-23T18:20:37.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.057+0000 7f83117fa640 1 -- 172.21.15.37:0/271893114 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f830c0bdec0 con 0x7f831419cfe0 2024-04-23T18:20:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:37.051 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:29.010136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:20:37.051 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-04-23T18:20:37.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f82fc0a3db0 msgr2=0x7f82fc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f82fc0a3db0 0x7f82fc0a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f83040058c0 tx=0x7f830400a4c0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 msgr2=0x7f831419d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 0x7f831419d490 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f830c005cc0 tx=0x7f830c003ba0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 shutdown_connections 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f82fc0a3db0 0x7f82fc0a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f831419cfe0 0x7f831419d490 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8314107280 0x7f8314198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 --2- 172.21.15.37:0/271893114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8314102240 0x7f8314198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 >> 172.21.15.37:0/271893114 conn(0x7f83140790e0 msgr2=0x7f83141000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 shutdown_connections 2024-04-23T18:20:37.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.059+0000 7f831a564640 1 -- 172.21.15.37:0/271893114 wait complete. 2024-04-23T18:20:37.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:36 smithi037 ceph-mon[29404]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:37.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:37.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:36 smithi172 ceph-mon[32456]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:37.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:36 smithi173 ceph-mon[32906]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 -- 172.21.15.37:0/3799499615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77581010a0 msgr2=0x7f7758101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 --2- 172.21.15.37:0/3799499615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77581010a0 0x7f7758101480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7748009900 tx=0x7f774802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 -- 172.21.15.37:0/3799499615 shutdown_connections 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 --2- 172.21.15.37:0/3799499615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 0x7f775810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 --2- 172.21.15.37:0/3799499615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7758101a50 0x7f7758105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 --2- 172.21.15.37:0/3799499615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77581010a0 0x7f7758101480 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.172+0000 7f775e607640 1 -- 172.21.15.37:0/3799499615 >> 172.21.15.37:0/3799499615 conn(0x7f7758077f50 msgr2=0x7f7758078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 -- 172.21.15.37:0/3799499615 shutdown_connections 2024-04-23T18:20:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 -- 172.21.15.37:0/3799499615 wait complete. 2024-04-23T18:20:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 Processor -- start 2024-04-23T18:20:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 -- start start 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77581010a0 0x7f77581995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 0x7f7758199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 0x7f77581a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7758111130 con 0x7f775819a030 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7758110fb0 con 0x7f7758106170 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.173+0000 7f775e607640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f77581112b0 con 0x7f77581010a0 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 0x7f77581a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 0x7f77581a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43060/0 (socket says 172.21.15.37:43060) 2024-04-23T18:20:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 -- 172.21.15.37:0/988532160 learned_addr learned my addr 172.21.15.37:0/988532160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f7757fff640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77581010a0 0x7f77581995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f77577fe640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 0x7f7758199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 -- 172.21.15.37:0/988532160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77581010a0 msgr2=0x7f77581995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77581010a0 0x7f77581995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 -- 172.21.15.37:0/988532160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 msgr2=0x7f7758199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 0x7f7758199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 -- 172.21.15.37:0/988532160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77581a4dd0 con 0x7f775819a030 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f77577fe640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 0x7f7758199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775cba4640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 0x7f77581a4630 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f775000b970 tx=0x7f775000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750019070 con 0x7f775819a030 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7750014440 con 0x7f775819a030 2024-04-23T18:20:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77581a50c0 con 0x7f775819a030 2024-04-23T18:20:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500075d0 con 0x7f775819a030 2024-04-23T18:20:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.174+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77581a55c0 con 0x7f775819a030 2024-04-23T18:20:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.175+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7758068d70 con 0x7f775819a030 2024-04-23T18:20:37.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.176+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f775001c070 con 0x7f775819a030 2024-04-23T18:20:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.177+0000 7f77557fa640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f77400a3de0 0x7f77400a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.177+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f77500cfdf0 con 0x7f775819a030 2024-04-23T18:20:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.177+0000 7f7757fff640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f77400a3de0 0x7f77400a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.177+0000 7f7757fff640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f77400a3de0 0x7f77400a62a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7748031440 tx=0x7f7748005e80 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.182+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f775009bdf0 con 0x7f775819a030 2024-04-23T18:20:37.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.371+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77581093d0 con 0x7f775819a030 2024-04-23T18:20:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.372+0000 7f77557fa640 1 -- 172.21.15.37:0/988532160 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23061 (secure 0 0 0) 0x7f77500a00f0 con 0x7f775819a030 2024-04-23T18:20:37.367 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:20:30.552696+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:37.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f77400a3de0 msgr2=0x7f77400a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f77400a3de0 0x7f77400a62a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7748031440 tx=0x7f7748005e80 comp rx=0 tx=0).stop 2024-04-23T18:20:37.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 msgr2=0x7f77581a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 0x7f77581a4630 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f775000b970 tx=0x7f775000be30 comp rx=0 tx=0).stop 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 shutdown_connections 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f77400a3de0 0x7f77400a62a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775819a030 0x7f77581a4630 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7758106170 0x7f7758199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 --2- 172.21.15.37:0/988532160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77581010a0 0x7f77581995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 >> 172.21.15.37:0/988532160 conn(0x7f7758077f50 msgr2=0x7f77580fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 shutdown_connections 2024-04-23T18:20:37.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.374+0000 7f775e607640 1 -- 172.21.15.37:0/988532160 wait complete. 2024-04-23T18:20:37.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:20:37.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 -- 172.21.15.37:0/4205153165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce7006a840 msgr2=0x7fce7006ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 --2- 172.21.15.37:0/4205153165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce7006a840 0x7fce7006ac20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fce60009900 tx=0x7fce6002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 -- 172.21.15.37:0/4205153165 shutdown_connections 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 --2- 172.21.15.37:0/4205153165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce70107270 0x7fce7010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 --2- 172.21.15.37:0/4205153165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006b1f0 0x7fce70106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 --2- 172.21.15.37:0/4205153165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce7006a840 0x7fce7006ac20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 -- 172.21.15.37:0/4205153165 >> 172.21.15.37:0/4205153165 conn(0x7fce70077eb0 msgr2=0x7fce700782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 -- 172.21.15.37:0/4205153165 shutdown_connections 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.491+0000 7fce75bba640 1 -- 172.21.15.37:0/4205153165 wait complete. 2024-04-23T18:20:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 Processor -- start 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 -- start start 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006a840 0x7fce70100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 0x7fce701013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce70101900 0x7fce701a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce701121d0 con 0x7fce70107270 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112050 con 0x7fce70101900 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce75bba640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112350 con 0x7fce7006a840 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.492+0000 7fce6effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 0x7fce701013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 0x7fce701013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43074/0 (socket says 172.21.15.37:43074) 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 -- 172.21.15.37:0/3978154962 learned_addr learned my addr 172.21.15.37:0/3978154962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6ffff640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce70101900 0x7fce701a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6f7fe640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006a840 0x7fce70100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 -- 172.21.15.37:0/3978154962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006a840 msgr2=0x7fce70100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006a840 0x7fce70100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 -- 172.21.15.37:0/3978154962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce70101900 msgr2=0x7fce701a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce70101900 0x7fce701a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 -- 172.21.15.37:0/3978154962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce701a6fe0 con 0x7fce70107270 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6f7fe640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006a840 0x7fce70100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6effd640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 0x7fce701013c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fce6400e9f0 tx=0x7fce6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400ccc0 con 0x7fce70107270 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640054d0 con 0x7fce70107270 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce701a72d0 con 0x7fce70107270 2024-04-23T18:20:37.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64010430 con 0x7fce70107270 2024-04-23T18:20:37.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.493+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a77d0 con 0x7fce70107270 2024-04-23T18:20:37.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.494+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce700691c0 con 0x7fce70107270 2024-04-23T18:20:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.495+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fce640046c0 con 0x7fce70107270 2024-04-23T18:20:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.496+0000 7fce6cff9640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fce580a3de0 0x7fce580a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.496+0000 7fce6f7fe640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fce580a3de0 0x7fce580a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.496+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fce640d0d40 con 0x7fce70107270 2024-04-23T18:20:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.497+0000 7fce6f7fe640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fce580a3de0 0x7fce580a62a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce60031440 tx=0x7fce60005a20 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.501+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce6409d280 con 0x7fce70107270 2024-04-23T18:20:37.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.685+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fce7010a4a0 con 0x7fce70107270 2024-04-23T18:20:37.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.685+0000 7fce6cff9640 1 -- 172.21.15.37:0/3978154962 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fce6409cb80 con 0x7fce70107270 2024-04-23T18:20:37.679 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:20:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fce580a3de0 msgr2=0x7fce580a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fce580a3de0 0x7fce580a62a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce60031440 tx=0x7fce60005a20 comp rx=0 tx=0).stop 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 msgr2=0x7fce701013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 0x7fce701013c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fce6400e9f0 tx=0x7fce6400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 shutdown_connections 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fce580a3de0 0x7fce580a62a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce70101900 0x7fce701a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce70107270 0x7fce701013c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 --2- 172.21.15.37:0/3978154962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce7006a840 0x7fce70100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 >> 172.21.15.37:0/3978154962 conn(0x7fce70077eb0 msgr2=0x7fce70108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 shutdown_connections 2024-04-23T18:20:37.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.687+0000 7fce75bba640 1 -- 172.21.15.37:0/3978154962 wait complete. 2024-04-23T18:20:37.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:20:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 -- 172.21.15.37:0/4223976909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c101090 msgr2=0x7f6d7c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 --2- 172.21.15.37:0/4223976909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c101090 0x7f6d7c101470 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009900 tx=0x7f6d6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 -- 172.21.15.37:0/4223976909 shutdown_connections 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 --2- 172.21.15.37:0/4223976909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d7c106160 0x7f6d7c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 --2- 172.21.15.37:0/4223976909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d7c101a40 0x7f6d7c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 --2- 172.21.15.37:0/4223976909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c101090 0x7f6d7c101470 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 -- 172.21.15.37:0/4223976909 >> 172.21.15.37:0/4223976909 conn(0x7f6d7c0771d0 msgr2=0x7f6d7c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 -- 172.21.15.37:0/4223976909 shutdown_connections 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.804+0000 7f6d823f7640 1 -- 172.21.15.37:0/4223976909 wait complete. 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 Processor -- start 2024-04-23T18:20:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 -- start start 2024-04-23T18:20:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d7c101090 0x7f6d7c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d7c106160 0x7f6d7c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 0x7f6d7c19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1121e0 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c112060 con 0x7f6d7c106160 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d823f7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c112360 con 0x7f6d7c101090 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d80994640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 0x7f6d7c19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d80994640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 0x7f6d7c19d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43086/0 (socket says 172.21.15.37:43086) 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d80994640 1 -- 172.21.15.37:0/2533258460 learned_addr learned my addr 172.21.15.37:0/2533258460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d7bfff640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d7c101090 0x7f6d7c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.805+0000 7f6d7b7fe640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d7c106160 0x7f6d7c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d80994640 1 -- 172.21.15.37:0/2533258460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d7c101090 msgr2=0x7f6d7c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d80994640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d7c101090 0x7f6d7c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d80994640 1 -- 172.21.15.37:0/2533258460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d7c106160 msgr2=0x7f6d7c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d80994640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d7c106160 0x7f6d7c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d80994640 1 -- 172.21.15.37:0/2533258460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c19dad0 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d80994640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 0x7f6d7c19d490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400cad0 tx=0x7f6d7400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400ae20 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d74005020 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c19ddc0 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400f6b0 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.806+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c1a5560 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.807+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7c102560 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.808+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6d740051d0 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.809+0000 7f6d797fa640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6d640a3db0 0x7f6d640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.809+0000 7f6d7bfff640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6d640a3db0 0x7f6d640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.809+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f6d740d0da0 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.809+0000 7f6d7bfff640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6d640a3db0 0x7f6d640a6270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031440 tx=0x7f6d6c005b50 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.813+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d74014030 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.999+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d7c199a70 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:37.999+0000 7f6d797fa640 1 -- 172.21.15.37:0/2533258460 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f6d7409d280 con 0x7f6d7c19cfe0 2024-04-23T18:20:37.993 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6d640a3db0 msgr2=0x7f6d640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6d640a3db0 0x7f6d640a6270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031440 tx=0x7f6d6c005b50 comp rx=0 tx=0).stop 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 msgr2=0x7f6d7c19d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 0x7f6d7c19d490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400cad0 tx=0x7f6d7400cf90 comp rx=0 tx=0).stop 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 shutdown_connections 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f6d640a3db0 0x7f6d640a6270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d7c19cfe0 0x7f6d7c19d490 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d7c106160 0x7f6d7c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 --2- 172.21.15.37:0/2533258460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d7c101090 0x7f6d7c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:37.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 >> 172.21.15.37:0/2533258460 conn(0x7f6d7c0771d0 msgr2=0x7f6d7c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:37.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 shutdown_connections 2024-04-23T18:20:37.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.001+0000 7f6d823f7640 1 -- 172.21.15.37:0/2533258460 wait complete. 2024-04-23T18:20:38.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:37 smithi037 ceph-mon[29404]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:38.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:37 smithi037 ceph-mon[29404]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 65 op/s 2024-04-23T18:20:38.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:37 smithi037 ceph-mon[29404]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:38.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:37 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/271893114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:37 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/988532160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:37 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3978154962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:38.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:20:38.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:37 smithi172 ceph-mon[32456]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:38.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:37 smithi172 ceph-mon[32456]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 65 op/s 2024-04-23T18:20:38.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:37 smithi172 ceph-mon[32456]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:38.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:37 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/271893114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:37 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/988532160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.088 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:37 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3978154962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:37 smithi173 ceph-mon[32906]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:37 smithi173 ceph-mon[32906]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 65 op/s 2024-04-23T18:20:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:37 smithi173 ceph-mon[32906]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:20:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:37 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/271893114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:37 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/988532160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:37 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3978154962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 -- 172.21.15.37:0/2291504415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c102b40 msgr2=0x7fc86c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 --2- 172.21.15.37:0/2291504415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c102b40 0x7fc86c106b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc860009900 tx=0x7fc86002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 -- 172.21.15.37:0/2291504415 shutdown_connections 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 --2- 172.21.15.37:0/2291504415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc86c107260 0x7fc86c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 --2- 172.21.15.37:0/2291504415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c102b40 0x7fc86c106b80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 --2- 172.21.15.37:0/2291504415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc86c102190 0x7fc86c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 -- 172.21.15.37:0/2291504415 >> 172.21.15.37:0/2291504415 conn(0x7fc86c0fd990 msgr2=0x7fc86c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 -- 172.21.15.37:0/2291504415 shutdown_connections 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.117+0000 7fc872729640 1 -- 172.21.15.37:0/2291504415 wait complete. 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 Processor -- start 2024-04-23T18:20:38.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 -- start start 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc86c102190 0x7fc86c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 0x7fc86c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc86c074cd0 0x7fc86c1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112180 con 0x7fc86c107260 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112000 con 0x7fc86c074cd0 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc872729640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112300 con 0x7fc86c102190 2024-04-23T18:20:38.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 0x7fc86c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 0x7fc86c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43100/0 (socket says 172.21.15.37:43100) 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 -- 172.21.15.37:0/491648396 learned_addr learned my addr 172.21.15.37:0/491648396 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86bfff640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc86c102190 0x7fc86c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 -- 172.21.15.37:0/491648396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc86c102190 msgr2=0x7fc86c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc86c102190 0x7fc86c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 -- 172.21.15.37:0/491648396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc86c074cd0 msgr2=0x7fc86c1a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc86c074cd0 0x7fc86c1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.118+0000 7fc86b7fe640 1 -- 172.21.15.37:0/491648396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c1a7070 con 0x7fc86c107260 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.119+0000 7fc86b7fe640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 0x7fc86c074790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc860032fd0 tx=0x7fc860004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.119+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86003f030 con 0x7fc86c107260 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.119+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c1a7300 con 0x7fc86c107260 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.119+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860004490 con 0x7fc86c107260 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.119+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860035070 con 0x7fc86c107260 2024-04-23T18:20:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.119+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a77f0 con 0x7fc86c107260 2024-04-23T18:20:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.120+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c0691c0 con 0x7fc86c107260 2024-04-23T18:20:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.121+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc860042030 con 0x7fc86c107260 2024-04-23T18:20:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.122+0000 7fc8697fa640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc8540a3db0 0x7fc8540a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.122+0000 7fc86bfff640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc8540a3db0 0x7fc8540a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.122+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc8600f13d0 con 0x7fc86c107260 2024-04-23T18:20:38.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.123+0000 7fc86bfff640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc8540a3db0 0x7fc8540a6270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005fd0 tx=0x7fc85c005850 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.126+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8600bd8b0 con 0x7fc86c107260 2024-04-23T18:20:38.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.311+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc86c10a490 con 0x7fc86c107260 2024-04-23T18:20:38.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.311+0000 7fc8697fa640 1 -- 172.21.15.37:0/491648396 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fc8600bd1b0 con 0x7fc86c107260 2024-04-23T18:20:38.305 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.313+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc8540a3db0 msgr2=0x7fc8540a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.313+0000 7fc872729640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc8540a3db0 0x7fc8540a6270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005fd0 tx=0x7fc85c005850 comp rx=0 tx=0).stop 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.313+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 msgr2=0x7fc86c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.313+0000 7fc872729640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 0x7fc86c074790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc860032fd0 tx=0x7fc860004270 comp rx=0 tx=0).stop 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 shutdown_connections 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc8540a3db0 0x7fc8540a6270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc86c074cd0 0x7fc86c1a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc86c107260 0x7fc86c074790 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 --2- 172.21.15.37:0/491648396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc86c102190 0x7fc86c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 >> 172.21.15.37:0/491648396 conn(0x7fc86c0fd990 msgr2=0x7fc86c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 shutdown_connections 2024-04-23T18:20:38.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.314+0000 7fc872729640 1 -- 172.21.15.37:0/491648396 wait complete. 2024-04-23T18:20:38.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:20:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 -- 172.21.15.37:0/197794006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4101110 msgr2=0x7f50f41014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 --2- 172.21.15.37:0/197794006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4101110 0x7f50f41014f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009900 tx=0x7f50e802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 -- 172.21.15.37:0/197794006 shutdown_connections 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 --2- 172.21.15.37:0/197794006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f4106190 0x7f50f410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 --2- 172.21.15.37:0/197794006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f4101ac0 0x7f50f4105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 --2- 172.21.15.37:0/197794006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4101110 0x7f50f41014f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 -- 172.21.15.37:0/197794006 >> 172.21.15.37:0/197794006 conn(0x7f50f4077fb0 msgr2=0x7f50f40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 -- 172.21.15.37:0/197794006 shutdown_connections 2024-04-23T18:20:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.426+0000 7f50f9fed640 1 -- 172.21.15.37:0/197794006 wait complete. 2024-04-23T18:20:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 Processor -- start 2024-04-23T18:20:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 -- start start 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f4101110 0x7f50f419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 0x7f50f419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41985a0 0x7f50f4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41110d0 con 0x7f50f4106190 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4110f50 con 0x7f50f4101110 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.427+0000 7f50f9fed640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4111250 con 0x7f50f41985a0 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 0x7f50f419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 0x7f50f419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43106/0 (socket says 172.21.15.37:43106) 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 -- 172.21.15.37:0/1737329625 learned_addr learned my addr 172.21.15.37:0/1737329625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f3fff640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41985a0 0x7f50f4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f37fe640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f4101110 0x7f50f419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 -- 172.21.15.37:0/1737329625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41985a0 msgr2=0x7f50f4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41985a0 0x7f50f4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 -- 172.21.15.37:0/1737329625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f4101110 msgr2=0x7f50f419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f4101110 0x7f50f419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 -- 172.21.15.37:0/1737329625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41990c0 con 0x7f50f4106190 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f2ffd640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 0x7f50f419e930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f50e400d770 tx=0x7f50e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4017070 con 0x7f50f4106190 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e4009c30 con 0x7f50f4106190 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41993b0 con 0x7f50f4106190 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400f620 con 0x7f50f4106190 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.428+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a5600 con 0x7f50f4106190 2024-04-23T18:20:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.429+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f40691c0 con 0x7f50f4106190 2024-04-23T18:20:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.431+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f50e401a070 con 0x7f50f4106190 2024-04-23T18:20:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.432+0000 7f50f0ff9640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f50dc0a3db0 0x7f50dc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.432+0000 7f50f37fe640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f50dc0a3db0 0x7f50dc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.432+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f50e40d0450 con 0x7f50f4106190 2024-04-23T18:20:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.432+0000 7f50f37fe640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f50dc0a3db0 0x7f50dc0a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50e8031440 tx=0x7f50e8005b50 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.435+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50e4009de0 con 0x7f50f4106190 2024-04-23T18:20:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.660+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50f4101b50 con 0x7f50f4106190 2024-04-23T18:20:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.661+0000 7f50f0ff9640 1 -- 172.21.15.37:0/1737329625 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f50e409c2e0 con 0x7f50f4106190 2024-04-23T18:20:38.655 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:38.655 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:29.010136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:20:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-04-23T18:20:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f50dc0a3db0 msgr2=0x7f50dc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f50dc0a3db0 0x7f50dc0a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50e8031440 tx=0x7f50e8005b50 comp rx=0 tx=0).stop 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 msgr2=0x7f50f419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 0x7f50f419e930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f50e400d770 tx=0x7f50e400dc30 comp rx=0 tx=0).stop 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 shutdown_connections 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f50dc0a3db0 0x7f50dc0a6270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41985a0 0x7f50f4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50f4106190 0x7f50f419e930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 --2- 172.21.15.37:0/1737329625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f4101110 0x7f50f419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 >> 172.21.15.37:0/1737329625 conn(0x7f50f4077fb0 msgr2=0x7f50f4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 shutdown_connections 2024-04-23T18:20:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.663+0000 7f50f9fed640 1 -- 172.21.15.37:0/1737329625 wait complete. 2024-04-23T18:20:38.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.776+0000 7fe9af38e640 1 -- 172.21.15.37:0/3448241437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a8101a50 msgr2=0x7fe9a8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- 172.21.15.37:0/3448241437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a8101a50 0x7fe9a8105a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe998009900 tx=0x7fe99802a410 comp rx=0 tx=0).stop 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- 172.21.15.37:0/3448241437 shutdown_connections 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- 172.21.15.37:0/3448241437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a8106170 0x7fe9a810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- 172.21.15.37:0/3448241437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a8101a50 0x7fe9a8105a90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- 172.21.15.37:0/3448241437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9a81010a0 0x7fe9a8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- 172.21.15.37:0/3448241437 >> 172.21.15.37:0/3448241437 conn(0x7fe9a8077f50 msgr2=0x7fe9a8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- 172.21.15.37:0/3448241437 shutdown_connections 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- 172.21.15.37:0/3448241437 wait complete. 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 Processor -- start 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- start start 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a81010a0 0x7fe9a8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9a8106170 0x7fe9a8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 0x7fe9a819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81121f0 con 0x7fe9a819cfd0 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8112070 con 0x7fe9a81010a0 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.777+0000 7fe9af38e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8112370 con 0x7fe9a8106170 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 0x7fe9a819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 0x7fe9a819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43120/0 (socket says 172.21.15.37:43120) 2024-04-23T18:20:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 -- 172.21.15.37:0/1549220719 learned_addr learned my addr 172.21.15.37:0/1549220719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad12a640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a81010a0 0x7fe9a8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ac929640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9a8106170 0x7fe9a8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 -- 172.21.15.37:0/1549220719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9a8106170 msgr2=0x7fe9a8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9a8106170 0x7fe9a8198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 -- 172.21.15.37:0/1549220719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a81010a0 msgr2=0x7fe9a8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a81010a0 0x7fe9a8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 -- 172.21.15.37:0/1549220719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a819dac0 con 0x7fe9a819cfd0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad12a640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a81010a0 0x7fe9a8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9ad92b640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 0x7fe9a819d480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000cce0 tx=0x7fe9a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000f450 con 0x7fe9a819cfd0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a819ddb0 con 0x7fe9a819cfd0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a000faa0 con 0x7fe9a819cfd0 2024-04-23T18:20:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0018b70 con 0x7fe9a819cfd0 2024-04-23T18:20:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.778+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a5570 con 0x7fe9a819cfd0 2024-04-23T18:20:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.779+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a8068d70 con 0x7fe9a819cfd0 2024-04-23T18:20:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.781+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fe9a0018d10 con 0x7fe9a819cfd0 2024-04-23T18:20:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.782+0000 7fe99e7fc640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9940a3db0 0x7fe9940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.782+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fe9a00d0ee0 con 0x7fe9a819cfd0 2024-04-23T18:20:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.782+0000 7fe9ad12a640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9940a3db0 0x7fe9940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.782+0000 7fe9ad12a640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9940a3db0 0x7fe9940a6270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40058c0 tx=0x7fe9a4005850 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.785+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9a0014030 con 0x7fe9a819cfd0 2024-04-23T18:20:38.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.975+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9a8101a50 con 0x7fe9a819cfd0 2024-04-23T18:20:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.977+0000 7fe99e7fc640 1 -- 172.21.15.37:0/1549220719 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23061 (secure 0 0 0) 0x7fe9a009d3c0 con 0x7fe9a819cfd0 2024-04-23T18:20:38.970 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:38.970 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:20:30.552696+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:20:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9940a3db0 msgr2=0x7fe9940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9940a3db0 0x7fe9940a6270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40058c0 tx=0x7fe9a4005850 comp rx=0 tx=0).stop 2024-04-23T18:20:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 msgr2=0x7fe9a819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 0x7fe9a819d480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000cce0 tx=0x7fe9a000a620 comp rx=0 tx=0).stop 2024-04-23T18:20:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 shutdown_connections 2024-04-23T18:20:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9940a3db0 0x7fe9940a6270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a819cfd0 0x7fe9a819d480 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9a8106170 0x7fe9a8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 --2- 172.21.15.37:0/1549220719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a81010a0 0x7fe9a8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 >> 172.21.15.37:0/1549220719 conn(0x7fe9a8077f50 msgr2=0x7fe9a80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 shutdown_connections 2024-04-23T18:20:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:38.979+0000 7fe9af38e640 1 -- 172.21.15.37:0/1549220719 wait complete. 2024-04-23T18:20:38.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2533258460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:38.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/491648396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:38.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:38 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1737329625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:38.984 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 605219 2024-04-23T18:20:38.984 DEBUG:tasks.quiescer:Yielding 2024-04-23T18:20:38.984 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T18:20:38.998 INFO:tasks.workunit:Pulling workunits from ref 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:20:38.998 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T18:20:38.998 INFO:tasks.workunit:timeout=3h 2024-04-23T18:20:38.998 INFO:tasks.workunit:cleanup=True 2024-04-23T18:20:38.999 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:38.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-23 18:18:41.528903417 +0000 2024-04-23T18:20:39.015 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-23 18:18:41.528903417 +0000 2024-04-23T18:20:39.016 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-23 18:20:28.311864041 +0000 2024-04-23T18:20:39.016 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-23 18:18:41.528903417 +0000 2024-04-23T18:20:39.016 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T18:20:39.016 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T18:20:39.087 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 -- 172.21.15.37:0/47725070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac102b70 msgr2=0x7f9dac106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 --2- 172.21.15.37:0/47725070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac102b70 0x7f9dac106bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009900 tx=0x7f9d9c02a410 comp rx=0 tx=0).stop 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 -- 172.21.15.37:0/47725070 shutdown_connections 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 --2- 172.21.15.37:0/47725070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac107290 0x7f9dac10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 --2- 172.21.15.37:0/47725070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac102b70 0x7f9dac106bb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 --2- 172.21.15.37:0/47725070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac1021c0 0x7f9dac1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 -- 172.21.15.37:0/47725070 >> 172.21.15.37:0/47725070 conn(0x7f9dac0793e0 msgr2=0x7f9dac0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 -- 172.21.15.37:0/47725070 shutdown_connections 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.110+0000 7f9db3691640 1 -- 172.21.15.37:0/47725070 wait complete. 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 Processor -- start 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 -- start start 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac1021c0 0x7f9dac074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac107290 0x7f9dac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 0x7f9dac1a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1122b0 con 0x7f9dac074cd0 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112130 con 0x7f9dac107290 2024-04-23T18:20:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db3691640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112430 con 0x7f9dac1021c0 2024-04-23T18:20:39.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2533258460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:39.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/491648396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:39.106 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:38 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1737329625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db1c2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 0x7f9dac1a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db1c2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 0x7f9dac1a6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43128/0 (socket says 172.21.15.37:43128) 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db1c2e640 1 -- 172.21.15.37:0/2396814895 learned_addr learned my addr 172.21.15.37:0/2396814895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db142d640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac1021c0 0x7f9dac074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db0c2c640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac107290 0x7f9dac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.111+0000 7f9db1c2e640 1 -- 172.21.15.37:0/2396814895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac1021c0 msgr2=0x7f9dac074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db1c2e640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac1021c0 0x7f9dac074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db1c2e640 1 -- 172.21.15.37:0/2396814895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac107290 msgr2=0x7f9dac074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db1c2e640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac107290 0x7f9dac074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db1c2e640 1 -- 172.21.15.37:0/2396814895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1a7160 con 0x7f9dac074cd0 2024-04-23T18:20:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db1c2e640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 0x7f9dac1a6ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9da400b9e0 tx=0x7f9da400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4019070 con 0x7f9dac074cd0 2024-04-23T18:20:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da4014440 con 0x7f9dac074cd0 2024-04-23T18:20:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1a7450 con 0x7f9dac074cd0 2024-04-23T18:20:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40075d0 con 0x7f9dac074cd0 2024-04-23T18:20:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.112+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a7920 con 0x7f9dac074cd0 2024-04-23T18:20:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.113+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac068d70 con 0x7f9dac074cd0 2024-04-23T18:20:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.114+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f9da401c070 con 0x7f9dac074cd0 2024-04-23T18:20:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.115+0000 7f9da27fc640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9d980a3db0 0x7f9d980a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.115+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f9da40d03c0 con 0x7f9dac074cd0 2024-04-23T18:20:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.115+0000 7f9db142d640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9d980a3db0 0x7f9d980a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.115+0000 7f9db142d640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9d980a3db0 0x7f9d980a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009990 tx=0x7f9da8009110 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.119+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9da409ca10 con 0x7f9dac074cd0 2024-04-23T18:20:39.119 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T18:20:39.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.343+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dac10a4c0 con 0x7f9dac074cd0 2024-04-23T18:20:39.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.344+0000 7f9da27fc640 1 -- 172.21.15.37:0/2396814895 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f9da409c310 con 0x7f9dac074cd0 2024-04-23T18:20:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:39.337 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:29.010136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:20:39.338 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-04-23T18:20:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:38 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2533258460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:20:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:38 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/491648396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:20:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:38 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1737329625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:39.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9d980a3db0 msgr2=0x7f9d980a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9d980a3db0 0x7f9d980a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009990 tx=0x7f9da8009110 comp rx=0 tx=0).stop 2024-04-23T18:20:39.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 msgr2=0x7f9dac1a6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 0x7f9dac1a6ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9da400b9e0 tx=0x7f9da400bde0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 shutdown_connections 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9d980a3db0 0x7f9d980a6270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dac074cd0 0x7f9dac1a6ac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac107290 0x7f9dac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 --2- 172.21.15.37:0/2396814895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac1021c0 0x7f9dac074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 >> 172.21.15.37:0/2396814895 conn(0x7f9dac0793e0 msgr2=0x7f9dac100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 shutdown_connections 2024-04-23T18:20:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.346+0000 7f9db3691640 1 -- 172.21.15.37:0/2396814895 wait complete. 2024-04-23T18:20:39.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:18:34.408115+0000', 'modified': '2024-04-23T18:20:29.010136+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24484, 'mds_2': 24479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2640036963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2640036963}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2640036963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2419231317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2419231317}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2419231317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6837/3035435067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3035435067}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3035435067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.172:6837/364708999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 364708999}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 364708999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.173:6839/496716250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 496716250}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 496716250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6839/2128854391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 2128854391}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 2128854391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24509, 24479, 24484]} 2024-04-23T18:20:39.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-23T18:20:39.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:20:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 -- 172.21.15.37:0/289914055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 msgr2=0x7f795810ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 --2- 172.21.15.37:0/289914055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f795810ea50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7950009ca0 tx=0x7f795002a950 comp rx=0 tx=0).stop 2024-04-23T18:20:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 -- 172.21.15.37:0/289914055 shutdown_connections 2024-04-23T18:20:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 --2- 172.21.15.37:0/289914055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f795810ea50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 --2- 172.21.15.37:0/289914055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103b50 0x7f795810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 --2- 172.21.15.37:0/289914055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7958103230 0x7f7958103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 -- 172.21.15.37:0/289914055 >> 172.21.15.37:0/289914055 conn(0x7f7958077f80 msgr2=0x7f7958078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 -- 172.21.15.37:0/289914055 shutdown_connections 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 -- 172.21.15.37:0/289914055 wait complete. 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 Processor -- start 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.501+0000 7f796042a640 1 -- start start 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103230 0x7f795819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7958103b50 0x7f795819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f79581985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112290 con 0x7f795810c600 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112110 con 0x7f7958103230 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112410 con 0x7f7958103b50 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f79581985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f79581985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43150/0 (socket says 172.21.15.37:43150) 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 -- 172.21.15.37:0/2537334105 learned_addr learned my addr 172.21.15.37:0/2537334105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e1c6640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103230 0x7f795819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 -- 172.21.15.37:0/2537334105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7958103b50 msgr2=0x7f795819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795d9c5640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7958103b50 0x7f795819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7958103b50 0x7f795819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 -- 172.21.15.37:0/2537334105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103230 msgr2=0x7f795819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103230 0x7f795819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 -- 172.21.15.37:0/2537334105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7958198d70 con 0x7f795810c600 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e1c6640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103230 0x7f795819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f795e9c7640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f79581985f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f795002ae70 tx=0x7f795002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f794f7fe640 1 -- 172.21.15.37:0/2537334105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950034040 con 0x7f795810c600 2024-04-23T18:20:39.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7958199000 con 0x7f795810c600 2024-04-23T18:20:39.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f794f7fe640 1 -- 172.21.15.37:0/2537334105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79500308f0 con 0x7f795810c600 2024-04-23T18:20:39.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f794f7fe640 1 -- 172.21.15.37:0/2537334105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500047f0 con 0x7f795810c600 2024-04-23T18:20:39.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.502+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a5300 con 0x7f795810c600 2024-04-23T18:20:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.505+0000 7f794f7fe640 1 -- 172.21.15.37:0/2537334105 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7950037020 con 0x7f795810c600 2024-04-23T18:20:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.505+0000 7f796321c740 1 Processor -- start 2024-04-23T18:20:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.505+0000 7f796321c740 1 -- start start 2024-04-23T18:20:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f796321c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9def190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f796321c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f796321c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9dfc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f796321c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fd9d54c50 con 0x564fd9deedb0 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f796321c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fd9cf1830 con 0x564fd9def760 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f796321c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564fd9d50c10 con 0x564fd9df9dd0 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f794f7fe640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f79440a3db0 0x7f79440a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9def190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9def190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43160/0 (socket says 172.21.15.37:43160) 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 -- 172.21.15.37:0/271483601 learned_addr learned my addr 172.21.15.37:0/271483601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795e9c7640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f79440a3db0 0x7f79440a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795e1c6640 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9dfc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f794f7fe640 1 -- 172.21.15.37:0/2537334105 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f79500f18c0 con 0x7f795810c600 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 -- 172.21.15.37:0/271483601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 msgr2=0x564fd9dfc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9dfc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 -- 172.21.15.37:0/271483601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 msgr2=0x564fd9df9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 -- 172.21.15.37:0/271483601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd9a672d0 con 0x564fd9deedb0 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795d9c5640 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9def190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f79480154a0 tx=0x7f7948009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778f7cdafc5feb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f795e9c7640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f79440a3db0 0x7f79440a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79500ef0d0 tx=0x7f7950087e10 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f794cbfe640 1 -- 172.21.15.37:0/271483601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948005020 con 0x564fd9deedb0 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f794cbfe640 1 -- 172.21.15.37:0/271483601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7948005670 con 0x564fd9deedb0 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.506+0000 7f794cbfe640 1 -- 172.21.15.37:0/271483601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480134c0 con 0x564fd9deedb0 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 -- 172.21.15.37:0/271483601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 msgr2=0x564fd9def190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9def190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f79480154a0 tx=0x7f7948009530 comp rx=0 tx=0).stop 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 -- 172.21.15.37:0/271483601 shutdown_connections 2024-04-23T18:20:39.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9dfc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 --2- 172.21.15.37:0/271483601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9def190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 -- 172.21.15.37:0/271483601 >> 172.21.15.37:0/271483601 conn(0x564fd9deb070 msgr2=0x564fd9deb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 -- 172.21.15.37:0/271483601 shutdown_connections 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.507+0000 7f796321c740 1 -- 172.21.15.37:0/271483601 wait complete. 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 -- start start 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 Processor -- start 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9df6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fd9dfcce0 con 0x564fd9deedb0 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fd9cef470 con 0x564fd9def760 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564fd9dfce60 con 0x564fd9df9dd0 2024-04-23T18:20:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9df6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f796321c740 10 client.0 _ll_register_callbacks cb 0x564fd9d14f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9df6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43174/0 (socket says 172.21.15.37:43174) 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 -- 172.21.15.37:0/2155697664 learned_addr learned my addr 172.21.15.37:0/2155697664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795e1c6640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 -- 172.21.15.37:0/2155697664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 msgr2=0x564fd9df76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 -- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 msgr2=0x564fd9df71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795d9c5640 1 -- 172.21.15.37:0/2155697664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd9b316b0 con 0x564fd9deedb0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.508+0000 7f795e1c6640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f795d9c5640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9df6c70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7948015470 tx=0x7f79480097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ccf670d75b6bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f7941ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f7941ffb640 1 -- 172.21.15.37:0/2155697664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794802a060 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f7941ffb640 1 -- 172.21.15.37:0/2155697664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f7948005200 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7948005670 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f7941ffb640 1 -- 172.21.15.37:0/2155697664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f79480049d0 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 10 client.15486 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564fd9e08f50 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f7941ffb640 1 -- 172.21.15.37:0/2155697664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7948005670 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f7941ffb640 1 -- 172.21.15.37:0/2155697664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f794800c030 con 0x564fd9deedb0 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 10 client.15486 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 10 client.15486 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 20 client.15486 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e0b230 0x564fd9e0c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 4 client.15486 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:39.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.509+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd9d62670 con 0x564fd9e0b230 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.510+0000 7f795e1c6640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e0b230 0x564fd9e0c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.510+0000 7f795e1c6640 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e0b230 0x564fd9e0c6a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef5a099b42f5211a server_cookie=692f6e572d2e1e49 in_seq=0 out_seq=0 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.510+0000 7f7941ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f7941ffb640 1 -- 172.21.15.37:0/2155697664 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fd9d62670 con 0x564fd9e0b230 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f7941ffb640 10 client.15486 handle_command_reply: tid=0 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 client.15486 shutdown 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 msgr2=0x564fd9df6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9df6c70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7948015470 tx=0x7f79480097d0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 shutdown_connections 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e0b230 0x564fd9e0c6a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9df71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f7941ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:39.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 --2- 172.21.15.37:0/2155697664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9df6c70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 >> 172.21.15.37:0/2155697664 conn(0x564fd9deb070 msgr2=0x564fd9df0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 shutdown_connections 2024-04-23T18:20:39.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 1 -- 172.21.15.37:0/2155697664 wait complete. 2024-04-23T18:20:39.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.511+0000 7f796321c740 20 client.15486 trim_cache size 0 max 16384 2024-04-23T18:20:39.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.529+0000 7f796321c740 1 Processor -- start 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 -- start start 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9deedb0 0x564fd9df1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 0x564fd9df5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fd9e0a9e0 con 0x564fd9def760 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fd9e0b0b0 con 0x564fd9deedb0 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f796321c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564fd9e0b780 con 0x564fd9df9dd0 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 0x564fd9df5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 0x564fd9df5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43176/0 (socket says 172.21.15.37:43176) 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 -- 172.21.15.37:0/3018819252 learned_addr learned my addr 172.21.15.37:0/3018819252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 -- 172.21.15.37:0/3018819252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 msgr2=0x564fd9df5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795d9c5640 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 -- 172.21.15.37:0/3018819252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9deedb0 msgr2=0x564fd9df1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9deedb0 0x564fd9df1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 -- 172.21.15.37:0/3018819252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd9dee350 con 0x564fd9def760 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795d9c5640 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f795e1c6640 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 0x564fd9df5540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f795400b9a0 tx=0x7f795400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeb9a8a6a3353c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.530+0000 7f7943fff640 1 -- 172.21.15.37:0/3018819252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795400f040 con 0x564fd9def760 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f7943fff640 1 -- 172.21.15.37:0/3018819252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795401e030 con 0x564fd9def760 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f7943fff640 1 -- 172.21.15.37:0/3018819252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954015740 con 0x564fd9def760 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 -- 172.21.15.37:0/3018819252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 msgr2=0x564fd9df5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 0x564fd9df5540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f795400b9a0 tx=0x7f795400bf00 comp rx=0 tx=0).stop 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 -- 172.21.15.37:0/3018819252 shutdown_connections 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9df5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9def760 0x564fd9df5540 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 --2- 172.21.15.37:0/3018819252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9deedb0 0x564fd9df1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 -- 172.21.15.37:0/3018819252 >> 172.21.15.37:0/3018819252 conn(0x564fd9deb070 msgr2=0x564fd9deb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 -- 172.21.15.37:0/3018819252 shutdown_connections 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 -- 172.21.15.37:0/3018819252 wait complete. 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 -- start start 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.531+0000 7f796321c740 1 Processor -- start 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9e14cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9e0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9e0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fd9e0c520 con 0x564fd9deedb0 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fd9e0be50 con 0x564fd9def760 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564fd9e0cbf0 con 0x564fd9df9dd0 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 10 client.0 _ll_register_callbacks cb 0x7f7948009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9e14cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9e14cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43178/0 (socket says 172.21.15.37:43178) 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 -- 172.21.15.37:0/621080713 learned_addr learned my addr 172.21.15.37:0/621080713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795e1c6640 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9e0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 -- 172.21.15.37:0/621080713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 msgr2=0x564fd9e0e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9e0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 -- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 msgr2=0x564fd9e0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9e0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 -- 172.21.15.37:0/621080713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd9e0ed90 con 0x564fd9deedb0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f795d9c5640 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9e14cb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7948015410 tx=0x7f7948009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f411e25e3405974f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f7941ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f7941ffb640 1 -- 172.21.15.37:0/621080713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794802a040 con 0x564fd9deedb0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f7941ffb640 1 -- 172.21.15.37:0/621080713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f7948004d70 con 0x564fd9deedb0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.532+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795401e030 con 0x564fd9deedb0 2024-04-23T18:20:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f7941ffb640 1 -- 172.21.15.37:0/621080713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7948010830 con 0x564fd9deedb0 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 10 client.15492 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564fd9e711e0 con 0x564fd9deedb0 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f7941ffb640 1 -- 172.21.15.37:0/621080713 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795401e030 con 0x564fd9deedb0 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f7941ffb640 1 -- 172.21.15.37:0/621080713 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f794800c070 con 0x564fd9deedb0 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 10 client.15492 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 10 client.15492 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 20 client.15492 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e75c10 0x564fd9e74920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 4 client.15492 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "status"}) -- 0x564fd9d5c4e0 con 0x564fd9e75c10 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.533+0000 7f795e1c6640 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e75c10 0x564fd9e74920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f7941ffb640 10 client.15492 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f795e1c6640 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e75c10 0x564fd9e74920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cc7becd08617406 server_cookie=c8cdae74cbbd69a3 in_seq=0 out_seq=0 2024-04-23T18:20:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f7941ffb640 1 -- 172.21.15.37:0/621080713 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564fd9d5c4e0 con 0x564fd9e75c10 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f7941ffb640 10 client.15492 handle_command_reply: tid=0 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 client.15492 shutdown 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 msgr2=0x564fd9e14cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9e14cb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7948015410 tx=0x7f7948009490 comp rx=0 tx=0).stop 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 shutdown_connections 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x564fd9e75c10 0x564fd9e74920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd9df9dd0 0x564fd9e0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f7941ffb640 0 client.15492 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd9def760 0x564fd9e0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 --2- 172.21.15.37:0/621080713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd9deedb0 0x564fd9e14cb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 >> 172.21.15.37:0/621080713 conn(0x564fd9deb070 msgr2=0x564fd9df05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 shutdown_connections 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 1 -- 172.21.15.37:0/621080713 wait complete. 2024-04-23T18:20:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.534+0000 7f796321c740 20 client.15492 trim_cache size 0 max 16384 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24479, 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 120.46040988599999, 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 141.98827193599999 2024-04-23T18:20:39.534 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f79440a3db0 msgr2=0x7f79440a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f79440a3db0 0x7f79440a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79500ef0d0 tx=0x7f7950087e10 comp rx=0 tx=0).stop 2024-04-23T18:20:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 msgr2=0x7f79581985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f79581985f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f795002ae70 tx=0x7f795002f5d0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 shutdown_connections 2024-04-23T18:20:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f79440a3db0 0x7f79440a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f795810c600 0x7f79581985f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7958103b50 0x7f795819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 --2- 172.21.15.37:0/2537334105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7958103230 0x7f795819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 >> 172.21.15.37:0/2537334105 conn(0x7f7958077f80 msgr2=0x7f79580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 shutdown_connections 2024-04-23T18:20:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.535+0000 7f796042a640 1 -- 172.21.15.37:0/2537334105 wait complete. 2024-04-23T18:20:39.538 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 -- 172.21.15.37:0/740923376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 msgr2=0x7fe74010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 --2- 172.21.15.37:0/740923376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74010ea50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe738009ca0 tx=0x7fe73802a950 comp rx=0 tx=0).stop 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 -- 172.21.15.37:0/740923376 shutdown_connections 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 --2- 172.21.15.37:0/740923376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74010ea50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 --2- 172.21.15.37:0/740923376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe740103b50 0x7fe74010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 --2- 172.21.15.37:0/740923376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 0x7fe740103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 -- 172.21.15.37:0/740923376 >> 172.21.15.37:0/740923376 conn(0x7fe740077f80 msgr2=0x7fe740078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 -- 172.21.15.37:0/740923376 shutdown_connections 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 -- 172.21.15.37:0/740923376 wait complete. 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.649+0000 7fe747ea7640 1 Processor -- start 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 -- start start 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 0x7fe740198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe740103b50 0x7fe740198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe740112260 con 0x7fe74010c600 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401120e0 con 0x7fe740103b50 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe747ea7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe7401123e0 con 0x7fe740103230 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43194/0 (socket says 172.21.15.37:43194) 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 -- 172.21.15.37:0/936833859 learned_addr learned my addr 172.21.15.37:0/936833859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe745c43640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 0x7fe740198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe745442640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe740103b50 0x7fe740198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 -- 172.21.15.37:0/936833859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 msgr2=0x7fe740198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 0x7fe740198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 -- 172.21.15.37:0/936833859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe740103b50 msgr2=0x7fe740198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe740103b50 0x7fe740198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 -- 172.21.15.37:0/936833859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74019d710 con 0x7fe74010c600 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe745c43640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 0x7fe740198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe746444640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74019cfe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe738009dc0 tx=0x7fe738003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.650+0000 7fe736ffd640 1 -- 172.21.15.37:0/936833859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738034040 con 0x7fe74010c600 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.651+0000 7fe736ffd640 1 -- 172.21.15.37:0/936833859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe738030850 con 0x7fe74010c600 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.651+0000 7fe736ffd640 1 -- 172.21.15.37:0/936833859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7380047f0 con 0x7fe74010c600 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.651+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74019d9a0 con 0x7fe74010c600 2024-04-23T18:20:39.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.651+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7401a5290 con 0x7fe74010c600 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.653+0000 7fe736ffd640 1 -- 172.21.15.37:0/936833859 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fe738037020 con 0x7fe74010c600 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.653+0000 7fe74abc4740 1 Processor -- start 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 -- start start 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 0x564a8e733190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e733760 0x564a8e73d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e740220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a8e698c50 con 0x564a8e732db0 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a8e635830 con 0x564a8e73ddd0 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe74abc4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564a8e694c10 con 0x564a8e733760 2024-04-23T18:20:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 0x564a8e733190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 0x564a8e733190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43200/0 (socket says 172.21.15.37:43200) 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 -- 172.21.15.37:0/3457041664 learned_addr learned my addr 172.21.15.37:0/3457041664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745c43640 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e733760 0x564a8e73d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe736ffd640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe72c0a3db0 0x7fe72c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 -- 172.21.15.37:0/3457041664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e733760 msgr2=0x564a8e73d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e733760 0x564a8e73d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 -- 172.21.15.37:0/3457041664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 msgr2=0x564a8e740220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e740220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe745442640 1 -- 172.21.15.37:0/3457041664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8e3ab2d0 con 0x564a8e732db0 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe746444640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe72c0a3db0 0x7fe72c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.654+0000 7fe736ffd640 1 -- 172.21.15.37:0/936833859 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fe7380f1920 con 0x7fe74010c600 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe745442640 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 0x564a8e733190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe730009b60 tx=0x7fe7300106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6291e645a703cdf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe72bfff640 1 -- 172.21.15.37:0/3457041664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730016070 con 0x564a8e732db0 2024-04-23T18:20:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe72bfff640 1 -- 172.21.15.37:0/3457041664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe730009e50 con 0x564a8e732db0 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe72bfff640 1 -- 172.21.15.37:0/3457041664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730011610 con 0x564a8e732db0 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe74abc4740 1 -- 172.21.15.37:0/3457041664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 msgr2=0x564a8e733190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe74abc4740 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 0x564a8e733190 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe730009b60 tx=0x7fe7300106e0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe746444640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe72c0a3db0 0x7fe72c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ef130 tx=0x7fe738088310 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe74abc4740 1 -- 172.21.15.37:0/3457041664 shutdown_connections 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe74abc4740 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e740220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe74abc4740 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e733760 0x564a8e73d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.655+0000 7fe74abc4740 1 --2- 172.21.15.37:0/3457041664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e732db0 0x564a8e733190 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- 172.21.15.37:0/3457041664 >> 172.21.15.37:0/3457041664 conn(0x564a8e72f070 msgr2=0x564a8e72f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- 172.21.15.37:0/3457041664 shutdown_connections 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- 172.21.15.37:0/3457041664 wait complete. 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- start start 2024-04-23T18:20:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 Processor -- start 2024-04-23T18:20:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e73ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e73b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e73b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a8e740d40 con 0x564a8e733760 2024-04-23T18:20:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a8e633470 con 0x564a8e73ddd0 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.656+0000 7fe74abc4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564a8e740ec0 con 0x564a8e732db0 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e73b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe74abc4740 10 client.0 _ll_register_callbacks cb 0x564a8e658f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e73b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43216/0 (socket says 172.21.15.37:43216) 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 -- 172.21.15.37:0/2821532752 learned_addr learned my addr 172.21.15.37:0/2821532752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745442640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e73ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745442640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e73b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 -- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 msgr2=0x564a8e73ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e73ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 -- 172.21.15.37:0/2821532752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 msgr2=0x564a8e73b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e73b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 -- 172.21.15.37:0/2821532752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8e4756b0 con 0x564a8e733760 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745442640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e73ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe745c43640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e73b170 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe73c010850 tx=0x7fe73c010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb23ca6368ed3f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe7297fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe7297fa640 1 -- 172.21.15.37:0/2821532752 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c01d060 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe7297fa640 1 -- 172.21.15.37:0/2821532752 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fe73c00eda0 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe730009e50 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe7297fa640 1 -- 172.21.15.37:0/2821532752 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe73c018430 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe74abc4740 10 client.15504 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.657+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564a8e74d030 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe7297fa640 1 -- 172.21.15.37:0/2821532752 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe730009e50 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe7297fa640 1 -- 172.21.15.37:0/2821532752 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe73c020710 con 0x564a8e733760 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe74abc4740 10 client.15504 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe74abc4740 10 client.15504 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe74abc4740 20 client.15504 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e74f6d0 0x564a8e750b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe74abc4740 4 client.15504 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a8e6a6670 con 0x564a8e74f6d0 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.658+0000 7fe745442640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e74f6d0 0x564a8e750b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe7297fa640 10 client.15504 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe745442640 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e74f6d0 0x564a8e750b40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc577169305399f9 server_cookie=ff6ae12e0623bb13 in_seq=0 out_seq=0 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe7297fa640 1 -- 172.21.15.37:0/2821532752 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a8e6a6670 con 0x564a8e74f6d0 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe7297fa640 10 client.15504 handle_command_reply: tid=0 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe74abc4740 1 client.15504 shutdown 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 msgr2=0x564a8e73b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.659+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e73b170 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe73c010850 tx=0x7fe73c010d10 comp rx=0 tx=0).stop 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 shutdown_connections 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e74f6d0 0x564a8e750b40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e73b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e73b170 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe7297fa640 0 client.15504 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2821532752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e73ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 >> 172.21.15.37:0/2821532752 conn(0x564a8e72f070 msgr2=0x564a8e734ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 shutdown_connections 2024-04-23T18:20:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 1 -- 172.21.15.37:0/2821532752 wait complete. 2024-04-23T18:20:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.660+0000 7fe74abc4740 20 client.15504 trim_cache size 0 max 16384 2024-04-23T18:20:39.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 Processor -- start 2024-04-23T18:20:39.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 -- start start 2024-04-23T18:20:39.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e74f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e7355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e759210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a8e74ee80 con 0x564a8e733760 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a8e74f550 con 0x564a8e73ddd0 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe74abc4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564a8e74fc20 con 0x564a8e732db0 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.678+0000 7fe745c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e7355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e7355f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43220/0 (socket says 172.21.15.37:43220) 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 -- 172.21.15.37:0/4063002168 learned_addr learned my addr 172.21.15.37:0/4063002168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745442640 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e759210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 -- 172.21.15.37:0/4063002168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 msgr2=0x564a8e74f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e74f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 -- 172.21.15.37:0/4063002168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 msgr2=0x564a8e759210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e759210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 -- 172.21.15.37:0/4063002168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8e736810 con 0x564a8e733760 2024-04-23T18:20:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745442640 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e759210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe745c43640 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e7355f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe73c001eb0 tx=0x7fe73c00bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc6d33d6d91ea86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe72b7fe640 1 -- 172.21.15.37:0/4063002168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00eba0 con 0x564a8e733760 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe72b7fe640 1 -- 172.21.15.37:0/4063002168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73c00ed40 con 0x564a8e733760 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.679+0000 7fe72b7fe640 1 -- 172.21.15.37:0/4063002168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c005230 con 0x564a8e733760 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- 172.21.15.37:0/4063002168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 msgr2=0x564a8e7355f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e7355f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe73c001eb0 tx=0x7fe73c00bc60 comp rx=0 tx=0).stop 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- 172.21.15.37:0/4063002168 shutdown_connections 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e73ddd0 0x564a8e759210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e7355f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- 172.21.15.37:0/4063002168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e732db0 0x564a8e74f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- 172.21.15.37:0/4063002168 >> 172.21.15.37:0/4063002168 conn(0x564a8e72f070 msgr2=0x564a8e739500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- 172.21.15.37:0/4063002168 shutdown_connections 2024-04-23T18:20:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- 172.21.15.37:0/4063002168 wait complete. 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- start start 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 Processor -- start 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e732db0 0x564a8e756a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e756f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e73ddd0 0x564a8e757480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a8e7509c0 con 0x564a8e733760 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a8e7502f0 con 0x564a8e732db0 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.680+0000 7fe74abc4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564a8e751090 con 0x564a8e73ddd0 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe74abc4740 10 client.0 _ll_register_callbacks cb 0x7fe73c00b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e756f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e756f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43222/0 (socket says 172.21.15.37:43222) 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 -- 172.21.15.37:0/2479687151 learned_addr learned my addr 172.21.15.37:0/2479687151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 -- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e73ddd0 msgr2=0x564a8e757480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e73ddd0 0x564a8e757480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 -- 172.21.15.37:0/2479687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e732db0 msgr2=0x564a8e756a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e732db0 0x564a8e756a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 -- 172.21.15.37:0/2479687151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8e7b88b0 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe745c43640 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e756f40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00bd10 tx=0x7fe73c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bb379beb966234 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe7297fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe7297fa640 1 -- 172.21.15.37:0/2479687151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c037060 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe73c00ed40 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe7297fa640 1 -- 172.21.15.37:0/2479687151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fe73c0205a0 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe7297fa640 1 -- 172.21.15.37:0/2479687151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe73c00e780 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe74abc4740 10 client.15510 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.681+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564a8e7b9ae0 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe7297fa640 1 -- 172.21.15.37:0/2479687151 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73c00ed40 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe7297fa640 1 -- 172.21.15.37:0/2479687151 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe73c02a550 con 0x564a8e733760 2024-04-23T18:20:39.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe74abc4740 10 client.15510 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe74abc4740 10 client.15510 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe74abc4740 20 client.15510 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e7be370 0x564a8e7bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe74abc4740 4 client.15510 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "status"}) -- 0x564a8e6a04e0 con 0x564a8e7be370 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe745442640 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e7be370 0x564a8e7bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe745442640 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e7be370 0x564a8e7bd080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b5a69f02c590ebb server_cookie=4e1281b089b82533 in_seq=0 out_seq=0 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.682+0000 7fe7297fa640 10 client.15510 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe7297fa640 1 -- 172.21.15.37:0/2479687151 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564a8e6a04e0 con 0x564a8e7be370 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe7297fa640 10 client.15510 handle_command_reply: tid=0 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 client.15510 shutdown 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 msgr2=0x564a8e756f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e756f40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00bd10 tx=0x7fe73c00b460 comp rx=0 tx=0).stop 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 shutdown_connections 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x564a8e7be370 0x564a8e7bd080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564a8e73ddd0 0x564a8e757480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a8e733760 0x564a8e756f40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe7297fa640 0 client.15510 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 --2- 172.21.15.37:0/2479687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8e732db0 0x564a8e756a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 >> 172.21.15.37:0/2479687151 conn(0x564a8e72f070 msgr2=0x564a8e749170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 shutdown_connections 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 1 -- 172.21.15.37:0/2479687151 wait complete. 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe74abc4740 20 client.15510 trim_cache size 0 max 16384 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:39.682 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24484, 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 121.61799864699999, 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 139.4353705 2024-04-23T18:20:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:39.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe72c0a3db0 msgr2=0x7fe72c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe72c0a3db0 0x7fe72c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ef130 tx=0x7fe738088310 comp rx=0 tx=0).stop 2024-04-23T18:20:39.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 msgr2=0x7fe74019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74019cfe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe738009dc0 tx=0x7fe738003f10 comp rx=0 tx=0).stop 2024-04-23T18:20:39.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 shutdown_connections 2024-04-23T18:20:39.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe72c0a3db0 0x7fe72c0a6270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe74010c600 0x7fe74019cfe0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe740103b50 0x7fe740198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.683+0000 7fe747ea7640 1 --2- 172.21.15.37:0/936833859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103230 0x7fe740198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.684+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 >> 172.21.15.37:0/936833859 conn(0x7fe740077f80 msgr2=0x7fe7400fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.684+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 shutdown_connections 2024-04-23T18:20:39.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.684+0000 7fe747ea7640 1 -- 172.21.15.37:0/936833859 wait complete. 2024-04-23T18:20:39.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:20:39.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 -- 172.21.15.37:0/2871264111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 msgr2=0x7ff5cc10ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 --2- 172.21.15.37:0/2871264111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc10ea50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009ca0 tx=0x7ff5c402a950 comp rx=0 tx=0).stop 2024-04-23T18:20:39.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 -- 172.21.15.37:0/2871264111 shutdown_connections 2024-04-23T18:20:39.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 --2- 172.21.15.37:0/2871264111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc10ea50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 --2- 172.21.15.37:0/2871264111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5cc103b50 0x7ff5cc10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 --2- 172.21.15.37:0/2871264111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5cc103230 0x7ff5cc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 -- 172.21.15.37:0/2871264111 >> 172.21.15.37:0/2871264111 conn(0x7ff5cc077f80 msgr2=0x7ff5cc078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 -- 172.21.15.37:0/2871264111 shutdown_connections 2024-04-23T18:20:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.797+0000 7ff5d3544640 1 -- 172.21.15.37:0/2871264111 wait complete. 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 Processor -- start 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 -- start start 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5cc103230 0x7ff5cc19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5cc103b50 0x7ff5cc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc112210 con 0x7ff5cc10c600 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc112090 con 0x7ff5cc103b50 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d3544640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc112390 con 0x7ff5cc103230 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43242/0 (socket says 172.21.15.37:43242) 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/3915099736 learned_addr learned my addr 172.21.15.37:0/3915099736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5cc103b50 0x7ff5cc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5cc103230 0x7ff5cc19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/3915099736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5cc103230 msgr2=0x7ff5cc19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5cc103230 0x7ff5cc19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/3915099736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5cc103b50 msgr2=0x7ff5cc19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5cc103b50 0x7ff5cc19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.798+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/3915099736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc198ef0 con 0x7ff5cc10c600 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.799+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc198630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4004af0 tx=0x7ff5c402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.799+0000 7ff5be7fc640 1 -- 172.21.15.37:0/3915099736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4034040 con 0x7ff5cc10c600 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.799+0000 7ff5be7fc640 1 -- 172.21.15.37:0/3915099736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c402fe80 con 0x7ff5cc10c600 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.799+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc199180 con 0x7ff5cc10c600 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.799+0000 7ff5be7fc640 1 -- 172.21.15.37:0/3915099736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40046a0 con 0x7ff5cc10c600 2024-04-23T18:20:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.799+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a54f0 con 0x7ff5cc10c600 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.801+0000 7ff5be7fc640 1 -- 172.21.15.37:0/3915099736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff5c4037020 con 0x7ff5cc10c600 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 Processor -- start 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 -- start start 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5be7fc640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff5b80a3e30 0x7ff5b80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9493190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe949d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ffe93f8c50 con 0x55ffe9492db0 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe9395830 con 0x55ffe9493760 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d6268740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ffe93f4c10 con 0x55ffe949ddd0 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9493190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff5b80a3e30 0x7ff5b80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9493190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43254/0 (socket says 172.21.15.37:43254) 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 -- 172.21.15.37:0/3628767842 learned_addr learned my addr 172.21.15.37:0/3628767842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5be7fc640 1 -- 172.21.15.37:0/3915099736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff5c40f1690 con 0x7ff5cc10c600 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe949d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 -- 172.21.15.37:0/3628767842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 msgr2=0x55ffe94a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 -- 172.21.15.37:0/3628767842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 msgr2=0x55ffe949d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe949d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d12e0640 1 -- 172.21.15.37:0/3628767842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe910b2d0 con 0x55ffe9492db0 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.802+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff5b80a3e30 0x7ff5b80a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80058a0 tx=0x7ff5c80127c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9493190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009cf0 tx=0x7ff5c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee158f15daa0ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5b77fe640 1 -- 172.21.15.37:0/3628767842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004230 con 0x55ffe9492db0 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5b77fe640 1 -- 172.21.15.37:0/3628767842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c00043d0 con 0x55ffe9492db0 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5b77fe640 1 -- 172.21.15.37:0/3628767842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0019770 con 0x55ffe9492db0 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d6268740 1 -- 172.21.15.37:0/3628767842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 msgr2=0x55ffe9493190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d6268740 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9493190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009cf0 tx=0x7ff5c0011810 comp rx=0 tx=0).stop 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d6268740 1 -- 172.21.15.37:0/3628767842 shutdown_connections 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d6268740 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d6268740 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe949d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.803+0000 7ff5d6268740 1 --2- 172.21.15.37:0/3628767842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9493190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- 172.21.15.37:0/3628767842 >> 172.21.15.37:0/3628767842 conn(0x55ffe948f070 msgr2=0x55ffe948f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- 172.21.15.37:0/3628767842 shutdown_connections 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- 172.21.15.37:0/3628767842 wait complete. 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- start start 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 Processor -- start 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe949ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9493760 0x55ffe949b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe949ddd0 0x55ffe949b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94a0d40 con 0x55ffe9492db0 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe9393470 con 0x55ffe949ddd0 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94a0ec0 con 0x55ffe9493760 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d6268740 10 client.0 _ll_register_callbacks cb 0x55ffe93b8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d12e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe949ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d12e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe949ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43258/0 (socket says 172.21.15.37:43258) 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.804+0000 7ff5d12e0640 1 -- 172.21.15.37:0/817209548 learned_addr learned my addr 172.21.15.37:0/817209548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9493760 0x55ffe949b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe949ddd0 0x55ffe949b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d12e0640 1 -- 172.21.15.37:0/817209548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9493760 msgr2=0x55ffe949b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9493760 0x55ffe949b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d12e0640 1 -- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe949ddd0 msgr2=0x55ffe949b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe949ddd0 0x55ffe949b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d12e0640 1 -- 172.21.15.37:0/817209548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe91d56b0 con 0x55ffe9492db0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe949ac40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730ab9a73e1c1f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/817209548 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0027070 con 0x55ffe9492db0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/817209548 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff5c0005240 con 0x55ffe9492db0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c00043d0 con 0x55ffe9492db0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/817209548 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff5c0033050 con 0x55ffe9492db0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d6268740 10 client.15522 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ffe94acfa0 con 0x55ffe9492db0 2024-04-23T18:20:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.805+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/817209548 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c00043d0 con 0x55ffe9492db0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/817209548 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff5c0019490 con 0x55ffe9492db0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d6268740 10 client.15522 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d6268740 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d6268740 20 client.15522 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d6268740 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe94af4b0 0x55ffe94b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d6268740 4 client.15522 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe9406670 con 0x55ffe94af4b0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe94af4b0 0x55ffe94b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe94af4b0 0x55ffe94b0920 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fde9bc88035e0d server_cookie=2168c92ac73b8b5a in_seq=0 out_seq=0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.806+0000 7ff5b4ff9640 10 client.15522 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/817209548 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffe9406670 con 0x55ffe94af4b0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5b4ff9640 10 client.15522 handle_command_reply: tid=0 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 client.15522 shutdown 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 msgr2=0x55ffe949ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe949ac40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c0003820 comp rx=0 tx=0).stop 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 shutdown_connections 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe94af4b0 0x55ffe94b0920 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe949ddd0 0x55ffe949b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9493760 0x55ffe949b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5b4ff9640 0 client.15522 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 --2- 172.21.15.37:0/817209548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe949ac40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 >> 172.21.15.37:0/817209548 conn(0x55ffe948f070 msgr2=0x55ffe9494ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 shutdown_connections 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.807+0000 7ff5d6268740 1 -- 172.21.15.37:0/817209548 wait complete. 2024-04-23T18:20:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.808+0000 7ff5d6268740 20 client.15522 trim_cache size 0 max 16384 2024-04-23T18:20:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 Processor -- start 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 -- start start 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9492db0 0x55ffe9495560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe9499da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 0x55ffe949b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94aec60 con 0x55ffe949ddd0 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94af330 con 0x55ffe9493760 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d6268740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94afa00 con 0x55ffe9492db0 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 0x55ffe949b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 0x55ffe949b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43268/0 (socket says 172.21.15.37:43268) 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/2805014866 learned_addr learned my addr 172.21.15.37:0/2805014866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/2805014866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9492db0 msgr2=0x55ffe9495560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9492db0 0x55ffe9495560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/2805014866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 msgr2=0x55ffe9499da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe9499da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 -- 172.21.15.37:0/2805014866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe948a9d0 con 0x55ffe949ddd0 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.826+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 0x55ffe949b4e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b9f50 tx=0x7ff5c4004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4214a43a2a027125 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5b6ffd640 1 -- 172.21.15.37:0/2805014866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c402c040 con 0x55ffe949ddd0 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5b6ffd640 1 -- 172.21.15.37:0/2805014866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c4004840 con 0x55ffe949ddd0 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5b6ffd640 1 -- 172.21.15.37:0/2805014866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40c61f0 con 0x55ffe949ddd0 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 -- 172.21.15.37:0/2805014866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 msgr2=0x55ffe949b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 0x55ffe949b4e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b9f50 tx=0x7ff5c4004140 comp rx=0 tx=0).stop 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 -- 172.21.15.37:0/2805014866 shutdown_connections 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe949ddd0 0x55ffe949b4e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe9499da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2805014866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe9492db0 0x55ffe9495560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 -- 172.21.15.37:0/2805014866 >> 172.21.15.37:0/2805014866 conn(0x55ffe948f070 msgr2=0x55ffe94a9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 -- 172.21.15.37:0/2805014866 shutdown_connections 2024-04-23T18:20:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.827+0000 7ff5d6268740 1 -- 172.21.15.37:0/2805014866 wait complete. 2024-04-23T18:20:39.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 -- start start 2024-04-23T18:20:39.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 Processor -- start 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9497940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe9497e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94b07a0 con 0x55ffe9492db0 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94b00d0 con 0x55ffe9493760 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ffe94b0e70 con 0x55ffe949ddd0 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d6268740 10 client.0 _ll_register_callbacks cb 0x7ff5c40eec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9497940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9497940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43274/0 (socket says 172.21.15.37:43274) 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 -- 172.21.15.37:0/2292175669 learned_addr learned my addr 172.21.15.37:0/2292175669 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d1ae1640 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 -- 172.21.15.37:0/2292175669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 msgr2=0x55ffe94983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 -- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 msgr2=0x55ffe9497e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe9497e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.828+0000 7ff5d12e0640 1 -- 172.21.15.37:0/2292175669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe94b6e80 con 0x55ffe9492db0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d12e0640 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9497940 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0003a10 tx=0x7ff5c0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8528f94e79ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/2292175669 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0030070 con 0x55ffe9492db0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/2292175669 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff5c0019a80 con 0x55ffe9492db0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffe94004e0 con 0x55ffe9492db0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/2292175669 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff5c0027040 con 0x55ffe9492db0 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 10 client.15528 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:39.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ffe9515770 con 0x55ffe9492db0 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/2292175669 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ffe94004e0 con 0x55ffe9492db0 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/2292175669 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff5c0023910 con 0x55ffe9492db0 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 10 client.15528 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 10 client.15528 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 20 client.15528 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe951a370 0x55ffe9519080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 4 client.15528 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.829+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "status"}) -- 0x55ffe937c9a0 con 0x55ffe951a370 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe951a370 0x55ffe9519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d0adf640 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe951a370 0x55ffe9519080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c50d4348c1c949 server_cookie=7052c7a9d41b709a in_seq=0 out_seq=0 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5b4ff9640 10 client.15528 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5b4ff9640 1 -- 172.21.15.37:0/2292175669 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ffe937c9a0 con 0x55ffe951a370 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5b4ff9640 10 client.15528 handle_command_reply: tid=0 2024-04-23T18:20:39.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 client.15528 shutdown 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 msgr2=0x55ffe9497940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9497940 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0003a10 tx=0x7ff5c0009f10 comp rx=0 tx=0).stop 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 shutdown_connections 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ffe951a370 0x55ffe9519080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe949ddd0 0x55ffe94983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe9493760 0x55ffe9497e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 --2- 172.21.15.37:0/2292175669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffe9492db0 0x55ffe9497940 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 >> 172.21.15.37:0/2292175669 conn(0x55ffe948f070 msgr2=0x55ffe94a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5b4ff9640 0 client.15528 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 shutdown_connections 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.830+0000 7ff5d6268740 1 -- 172.21.15.37:0/2292175669 wait complete. 2024-04-23T18:20:39.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d6268740 20 client.15528 trim_cache size 0 max 16384 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "af48a038-019c-11ef-bc93-c7b262605968", 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24509, 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 124.794278629, 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 23, 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 131.29849754099999 2024-04-23T18:20:39.830 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:39.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff5b80a3e30 msgr2=0x7ff5b80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff5b80a3e30 0x7ff5b80a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80058a0 tx=0x7ff5c80127c0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 msgr2=0x7ff5cc198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:39.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc198630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4004af0 tx=0x7ff5c402f5d0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 shutdown_connections 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff5b80a3e30 0x7ff5b80a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c600 0x7ff5cc198630 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5cc103b50 0x7ff5cc19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 --2- 172.21.15.37:0/3915099736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5cc103230 0x7ff5cc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 >> 172.21.15.37:0/3915099736 conn(0x7ff5cc077f80 msgr2=0x7ff5cc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 shutdown_connections 2024-04-23T18:20:39.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:39.831+0000 7ff5d3544640 1 -- 172.21.15.37:0/3915099736 wait complete. 2024-04-23T18:20:39.835 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-23T18:20:40.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:39 smithi173 ceph-mon[32906]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 75 op/s 2024-04-23T18:20:40.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1549220719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:40.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2396814895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:40.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:39 smithi037 ceph-mon[29404]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 75 op/s 2024-04-23T18:20:40.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1549220719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:40.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2396814895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:39 smithi172 ceph-mon[32456]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 75 op/s 2024-04-23T18:20:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1549220719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:20:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2396814895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:42.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:41 smithi173 ceph-mon[32906]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 58 op/s 2024-04-23T18:20:42.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:41 smithi037 ceph-mon[29404]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 58 op/s 2024-04-23T18:20:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:41 smithi172 ceph-mon[32456]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 58 op/s 2024-04-23T18:20:44.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:43 smithi173 ceph-mon[32906]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-23T18:20:44.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:43 smithi037 ceph-mon[29404]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-23T18:20:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:43 smithi172 ceph-mon[32456]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-23T18:20:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 -- 172.21.15.37:0/2913047509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec102b50 msgr2=0x7f8dec106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2913047509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec102b50 0x7f8dec106b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de002a410 comp rx=0 tx=0).stop 2024-04-23T18:20:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 -- 172.21.15.37:0/2913047509 shutdown_connections 2024-04-23T18:20:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2913047509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dec107270 0x7f8dec10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2913047509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec102b50 0x7f8dec106b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2913047509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dec1021a0 0x7f8dec102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 -- 172.21.15.37:0/2913047509 >> 172.21.15.37:0/2913047509 conn(0x7f8dec0fda00 msgr2=0x7f8dec0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 -- 172.21.15.37:0/2913047509 shutdown_connections 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.013+0000 7f8df1c05640 1 -- 172.21.15.37:0/2913047509 wait complete. 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.014+0000 7f8df1c05640 1 Processor -- start 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.014+0000 7f8df1c05640 1 -- start start 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.014+0000 7f8df1c05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 0x7f8dec1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.014+0000 7f8df1c05640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dec107270 0x7f8dec198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.014+0000 7f8df1c05640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dec19cf90 0x7f8dec19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.014+0000 7f8df1c05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec112190 con 0x7f8dec1021a0 2024-04-23T18:20:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8df1c05640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec112010 con 0x7f8dec107270 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8df1c05640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec112310 con 0x7f8dec19cf90 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 0x7f8dec1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 0x7f8dec1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50942/0 (socket says 172.21.15.37:50942) 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 -- 172.21.15.37:0/2878847835 learned_addr learned my addr 172.21.15.37:0/2878847835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deaffd640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dec107270 0x7f8dec198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8debfff640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dec19cf90 0x7f8dec19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 -- 172.21.15.37:0/2878847835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dec19cf90 msgr2=0x7f8dec19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dec19cf90 0x7f8dec19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 -- 172.21.15.37:0/2878847835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dec107270 msgr2=0x7f8dec198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dec107270 0x7f8dec198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 -- 172.21.15.37:0/2878847835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec19da80 con 0x7f8dec1021a0 2024-04-23T18:20:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 0x7f8dec1986d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00d950 tx=0x7f8ddc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8de8ff9640 1 -- 172.21.15.37:0/2878847835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc017070 con 0x7f8dec1021a0 2024-04-23T18:20:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec19dd70 con 0x7f8dec1021a0 2024-04-23T18:20:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8de8ff9640 1 -- 172.21.15.37:0/2878847835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ddc009c30 con 0x7f8dec1021a0 2024-04-23T18:20:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8de8ff9640 1 -- 172.21.15.37:0/2878847835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00f620 con 0x7f8dec1021a0 2024-04-23T18:20:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.015+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec1a5530 con 0x7f8dec1021a0 2024-04-23T18:20:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.018+0000 7f8de8ff9640 1 -- 172.21.15.37:0/2878847835 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8ddc010460 con 0x7f8dec1021a0 2024-04-23T18:20:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.018+0000 7f8df49f8740 1 Processor -- start 2024-04-23T18:20:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.018+0000 7f8df49f8740 1 -- start start 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dd93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd9e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8df49f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557a7dcf8970 con 0x557a7dd9e800 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8df49f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a7dc954a0 con 0x557a7dd93620 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8df49f8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557a7dcf4930 con 0x557a7dd93f40 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8de8ff9640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8dd40a3e60 0x7f8dd40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50956/0 (socket says 172.21.15.37:50956) 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 -- 172.21.15.37:0/1999817775 learned_addr learned my addr 172.21.15.37:0/1999817775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8debfff640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8dd40a3e60 0x7f8dd40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8de8ff9640 1 -- 172.21.15.37:0/2878847835 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f8ddc0d0630 con 0x7f8dec1021a0 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8debfff640 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dd93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd9e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 -- 172.21.15.37:0/1999817775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 msgr2=0x557a7dd9e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd9e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 -- 172.21.15.37:0/1999817775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 msgr2=0x557a7dd93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dd93a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 -- 172.21.15.37:0/1999817775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7da0b210 con 0x557a7dd9e800 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.019+0000 7f8deaffd640 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda0bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8dec06b5d0 tx=0x7f8de0032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63bf21a9a3af766 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.020+0000 7f8debfff640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8dd40a3e60 0x7f8dd40a6320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8de4005cc0 tx=0x7f8de400a340 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.020+0000 7f8dd1ffb640 1 -- 172.21.15.37:0/1999817775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0040070 con 0x557a7dd9e800 2024-04-23T18:20:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.020+0000 7f8dd1ffb640 1 -- 172.21.15.37:0/1999817775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de00316a0 con 0x557a7dd9e800 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.020+0000 7f8dd1ffb640 1 -- 172.21.15.37:0/1999817775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0035040 con 0x557a7dd9e800 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 -- 172.21.15.37:0/1999817775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 msgr2=0x557a7dda0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda0bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8dec06b5d0 tx=0x7f8de0032df0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 -- 172.21.15.37:0/1999817775 shutdown_connections 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda0bf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd9e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1999817775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dd93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 -- 172.21.15.37:0/1999817775 >> 172.21.15.37:0/1999817775 conn(0x557a7dd8fb30 msgr2=0x557a7dd8ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 -- 172.21.15.37:0/1999817775 shutdown_connections 2024-04-23T18:20:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 -- 172.21.15.37:0/1999817775 wait complete. 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 -- start start 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.021+0000 7f8df49f8740 1 Processor -- start 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7dd9e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93f40 0x557a7dd9b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd9e800 0x557a7dd9b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557a7dda13d0 con 0x557a7dd93620 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a7dc930b0 con 0x557a7dd93f40 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557a7dda1550 con 0x557a7dd9e800 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8df49f8740 10 client.0 _ll_register_callbacks cb 0x557a7dc80d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7dd9e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7dd9e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50960/0 (socket says 172.21.15.37:50960) 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deb7fe640 1 -- 172.21.15.37:0/4029000660 learned_addr learned my addr 172.21.15.37:0/4029000660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93f40 0x557a7dd9b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8debfff640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd9e800 0x557a7dd9b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 -- 172.21.15.37:0/4029000660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd9e800 msgr2=0x557a7dd9b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd9e800 0x557a7dd9b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 -- 172.21.15.37:0/4029000660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 msgr2=0x557a7dd9e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7dd9e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 -- 172.21.15.37:0/4029000660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7dad7620 con 0x557a7dd93f40 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7dd9e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.022+0000 7f8deaffd640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93f40 0x557a7dd9b3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de0009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279a75667cba0f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8dcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:20:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8dcaffd640 1 -- 172.21.15.37:0/4029000660 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0035070 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de00316a0 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8dcaffd640 1 -- 172.21.15.37:0/4029000660 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f8de0004230 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8dcaffd640 1 -- 172.21.15.37:0/4029000660 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8de0055050 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 10 client.25145 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557a7ddad270 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8dcaffd640 1 -- 172.21.15.37:0/4029000660 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de00316a0 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8dcaffd640 1 -- 172.21.15.37:0/4029000660 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8de0004750 con 0x557a7dd93f40 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 10 client.25145 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 10 client.25145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 20 client.25145 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7ddae5c0 0x557a7ddb09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 4 client.25145 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.023+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a7dd06390 con 0x557a7ddae5c0 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.024+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7ddae5c0 0x557a7ddb09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.024+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7ddae5c0 0x557a7ddb09b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c217ac66382c80a server_cookie=1581eb00b2cfc901 in_seq=0 out_seq=0 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.024+0000 7f8dcaffd640 10 client.25145 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8dcaffd640 1 -- 172.21.15.37:0/4029000660 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a7dd06390 con 0x557a7ddae5c0 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8dcaffd640 10 client.25145 handle_command_reply: tid=0 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 client.25145 shutdown 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93f40 msgr2=0x557a7dd9b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93f40 0x557a7dd9b3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de0009520 comp rx=0 tx=0).stop 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 shutdown_connections 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7ddae5c0 0x557a7ddb09b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd9e800 0x557a7dd9b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93f40 0x557a7dd9b3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8dcaffd640 0 client.25145 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 --2- 172.21.15.37:0/4029000660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7dd9e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 >> 172.21.15.37:0/4029000660 conn(0x557a7dd8fb30 msgr2=0x557a7dd95860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 shutdown_connections 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 1 -- 172.21.15.37:0/4029000660 wait complete. 2024-04-23T18:20:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.025+0000 7f8df49f8740 20 client.25145 trim_cache size 0 max 16384 2024-04-23T18:20:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.043+0000 7f8df49f8740 1 Processor -- start 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.043+0000 7f8df49f8740 1 -- start start 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.043+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7ddaa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd9e800 0x557a7dd9dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8df49f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1b02d0 con 0x557a7dd93620 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8df49f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1adba0 con 0x557a7dd9e800 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8df49f8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ab9f0 con 0x557a7dd93f40 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7ddaa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7ddaa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50970/0 (socket says 172.21.15.37:50970) 2024-04-23T18:20:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 -- 172.21.15.37:0/1146894284 learned_addr learned my addr 172.21.15.37:0/1146894284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deaffd640 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8debfff640 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd9e800 0x557a7dd9dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 -- 172.21.15.37:0/1146894284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 msgr2=0x557a7dd96af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd96af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 -- 172.21.15.37:0/1146894284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd9e800 msgr2=0x557a7dd9dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd9e800 0x557a7dd9dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 -- 172.21.15.37:0/1146894284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7dd9a370 con 0x557a7dd93620 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8debfff640 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd9e800 0x557a7dd9dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7ddaa290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc09c110 tx=0x7f8ddc097fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbddf3d2e22722b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8dd17fa640 1 -- 172.21.15.37:0/1146894284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc085070 con 0x557a7dd93620 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8dd17fa640 1 -- 172.21.15.37:0/1146894284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ddc0a07c0 con 0x557a7dd93620 2024-04-23T18:20:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.044+0000 7f8dd17fa640 1 -- 172.21.15.37:0/1146894284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc020040 con 0x557a7dd93620 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 -- 172.21.15.37:0/1146894284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 msgr2=0x557a7ddaa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7ddaa290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc09c110 tx=0x7f8ddc097fd0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 -- 172.21.15.37:0/1146894284 shutdown_connections 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd9e800 0x557a7dd9dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dd96af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1146894284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd93620 0x557a7ddaa290 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 -- 172.21.15.37:0/1146894284 >> 172.21.15.37:0/1146894284 conn(0x557a7dd8fb30 msgr2=0x557a7dd8ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 -- 172.21.15.37:0/1146894284 shutdown_connections 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.045+0000 7f8df49f8740 1 -- 172.21.15.37:0/1146894284 wait complete. 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 -- start start 2024-04-23T18:20:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 Processor -- start 2024-04-23T18:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dda67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dda6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ac060 con 0x557a7dd9e800 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ad4d0 con 0x557a7dd93620 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ac730 con 0x557a7dd93f40 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8df49f8740 10 client.0 _ll_register_callbacks cb 0x7f8de0032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8debfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8debfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50976/0 (socket says 172.21.15.37:50976) 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8debfff640 1 -- 172.21.15.37:0/1495131138 learned_addr learned my addr 172.21.15.37:0/1495131138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dda67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.046+0000 7f8deaffd640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dda6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8debfff640 1 -- 172.21.15.37:0/1495131138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 msgr2=0x557a7dda6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8debfff640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dda6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8debfff640 1 -- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 msgr2=0x557a7dda67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8debfff640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dda67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8debfff640 1 -- 172.21.15.37:0/1495131138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7dda2570 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8debfff640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda1d70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e350 tx=0x7f8de4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d766bcbdd64b365c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8dcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8dcaffd640 1 -- 172.21.15.37:0/1495131138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4010dc0 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ddc0a07c0 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8dcaffd640 1 -- 172.21.15.37:0/1495131138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f8de4004b20 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8dcaffd640 1 -- 172.21.15.37:0/1495131138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8de4004d20 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8df49f8740 10 client.15546 fetch_fsmap learned FSMap version 23 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557a7de14500 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.047+0000 7f8dcaffd640 1 -- 172.21.15.37:0/1495131138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ddc0a07c0 con 0x557a7dd9e800 2024-04-23T18:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8dcaffd640 1 -- 172.21.15.37:0/1495131138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8de4027070 con 0x557a7dd9e800 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8df49f8740 10 client.15546 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8df49f8740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8df49f8740 20 client.15546 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7de14d00 0x557a7de17120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8df49f8740 4 client.15546 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a7dd00200 con 0x557a7de14d00 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7de14d00 0x557a7de17120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8deb7fe640 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7de14d00 0x557a7de17120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce215a41a6d6ff5c server_cookie=f7b648a5e1e1c236 in_seq=0 out_seq=0 2024-04-23T18:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.048+0000 7f8dcaffd640 10 client.15546 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8dcaffd640 1 -- 172.21.15.37:0/1495131138 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a7dd00200 con 0x557a7de14d00 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8dcaffd640 10 client.15546 handle_command_reply: tid=0 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 client.15546 shutdown 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 msgr2=0x557a7dda1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda1d70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e350 tx=0x7f8de4004a40 comp rx=0 tx=0).stop 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 shutdown_connections 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557a7de14d00 0x557a7de17120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557a7dd9e800 0x557a7dda1d70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557a7dd93f40 0x557a7dda6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8dcaffd640 0 client.15546 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 --2- 172.21.15.37:0/1495131138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a7dd93620 0x557a7dda67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 >> 172.21.15.37:0/1495131138 conn(0x557a7dd8fb30 msgr2=0x557a7dd8ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 shutdown_connections 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 1 -- 172.21.15.37:0/1495131138 wait complete. 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.060+0000 7f8df49f8740 20 client.15546 trim_cache size 0 max 16384 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:20:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:20:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:20:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8dd40a3e60 msgr2=0x7f8dd40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8dd40a3e60 0x7f8dd40a6320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8de4005cc0 tx=0x7f8de400a340 comp rx=0 tx=0).stop 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 msgr2=0x7f8dec1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 0x7f8dec1986d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00d950 tx=0x7f8ddc00de10 comp rx=0 tx=0).stop 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 shutdown_connections 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8dd40a3e60 0x7f8dd40a6320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dec19cf90 0x7f8dec19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dec107270 0x7f8dec198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 --2- 172.21.15.37:0/2878847835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dec1021a0 0x7f8dec1986d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 >> 172.21.15.37:0/2878847835 conn(0x7f8dec0fda00 msgr2=0x7f8dec0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 shutdown_connections 2024-04-23T18:20:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:46.061+0000 7f8df1c05640 1 -- 172.21.15.37:0/2878847835 wait complete. 2024-04-23T18:20:46.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:45 smithi173 ceph-mon[32906]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-23T18:20:46.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:45 smithi037 ceph-mon[29404]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-23T18:20:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:45 smithi172 ceph-mon[32456]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-23T18:20:48.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:47 smithi037 ceph-mon[29404]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-23T18:20:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:47 smithi172 ceph-mon[32456]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-23T18:20:48.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:47 smithi173 ceph-mon[32906]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-23T18:20:50.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:49 smithi037 ceph-mon[29404]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-04-23T18:20:50.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:49 smithi172 ceph-mon[32456]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-04-23T18:20:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:49 smithi173 ceph-mon[32906]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 41 KiB/s wr, 26 op/s 2024-04-23T18:20:52.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:51 smithi037 ceph-mon[29404]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:20:52.157 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:51 smithi173 ceph-mon[32906]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:20:52.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:51 smithi172 ceph-mon[32456]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:20:54.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:53 smithi037 ceph-mon[29404]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-23T18:20:54.139 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:53 smithi172 ceph-mon[32456]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-23T18:20:54.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:53 smithi173 ceph-mon[32906]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-23T18:20:54.835 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:20:54.836 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.3 2024-04-23T18:20:54.836 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:20:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 -- 172.21.15.37:0/2239388454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc1010a0 msgr2=0x7f9ffc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 --2- 172.21.15.37:0/2239388454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc1010a0 0x7f9ffc101480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009ca0 tx=0x7f9fec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 -- 172.21.15.37:0/2239388454 shutdown_connections 2024-04-23T18:20:54.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 --2- 172.21.15.37:0/2239388454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ffc106170 0x7f9ffc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:54.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 --2- 172.21.15.37:0/2239388454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc101a50 0x7f9ffc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:54.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 --2- 172.21.15.37:0/2239388454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc1010a0 0x7f9ffc101480 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:54.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 -- 172.21.15.37:0/2239388454 >> 172.21.15.37:0/2239388454 conn(0x7f9ffc077f50 msgr2=0x7f9ffc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:54.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 -- 172.21.15.37:0/2239388454 shutdown_connections 2024-04-23T18:20:54.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.954+0000 7fa000b51640 1 -- 172.21.15.37:0/2239388454 wait complete. 2024-04-23T18:20:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 Processor -- start 2024-04-23T18:20:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 -- start start 2024-04-23T18:20:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ffc1010a0 0x7f9ffc199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:54.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc106170 0x7f9ffc199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 0x7f9ffc1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1110e0 con 0x7f9ffc19a050 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc110f60 con 0x7f9ffc106170 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7fa000b51640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc111260 con 0x7f9ffc1010a0 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7f9ffad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 0x7f9ffc1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7f9ffad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 0x7f9ffc1a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48084/0 (socket says 172.21.15.37:48084) 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.955+0000 7f9ffad9d640 1 -- 172.21.15.37:0/3754776641 learned_addr learned my addr 172.21.15.37:0/3754776641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffa59c640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ffc1010a0 0x7f9ffc199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ff9d9b640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc106170 0x7f9ffc199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffad9d640 1 -- 172.21.15.37:0/3754776641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ffc1010a0 msgr2=0x7f9ffc199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffad9d640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ffc1010a0 0x7f9ffc199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffad9d640 1 -- 172.21.15.37:0/3754776641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc106170 msgr2=0x7f9ffc199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffad9d640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc106170 0x7f9ffc199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:54.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffad9d640 1 -- 172.21.15.37:0/3754776641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc1a4e00 con 0x7f9ffc19a050 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ff9d9b640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc106170 0x7f9ffc199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9ffad9d640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 0x7f9ffc1a4600 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d750 tx=0x7f9ff400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4017070 con 0x7f9ffc19a050 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff40054d0 con 0x7f9ffc19a050 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc1a50f0 con 0x7f9ffc19a050 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4004c60 con 0x7f9ffc19a050 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.956+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc1a5640 con 0x7f9ffc19a050 2024-04-23T18:20:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.957+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc068d70 con 0x7f9ffc19a050 2024-04-23T18:20:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.958+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f9ff4009bd0 con 0x7f9ffc19a050 2024-04-23T18:20:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.959+0000 7f9feb7fe640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9fe40a3db0 0x7f9fe40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.959+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f9ff40d03d0 con 0x7f9ffc19a050 2024-04-23T18:20:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.959+0000 7f9ffa59c640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9fe40a3db0 0x7f9fe40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.960+0000 7f9ffa59c640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9fe40a3db0 0x7f9fe40a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9fec000f80 tx=0x7f9fec005ef0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:54.965+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ff409c8b0 con 0x7f9ffc19a050 2024-04-23T18:20:55.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:54 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:20:55.155 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:54 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:20:55.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.227+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ffc1093a0 con 0x7f9ffc19a050 2024-04-23T18:20:55.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.228+0000 7f9feb7fe640 1 -- 172.21.15.37:0/3754776641 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11876 (secure 0 0 0) 0x7f9ff409c1b0 con 0x7f9ffc19a050 2024-04-23T18:20:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:20:55.221 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:53.537980+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24484,24509,24479]},"id":1}]} 2024-04-23T18:20:55.222 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9fe40a3db0 msgr2=0x7f9fe40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9fe40a3db0 0x7f9fe40a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9fec000f80 tx=0x7f9fec005ef0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 msgr2=0x7f9ffc1a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 0x7f9ffc1a4600 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d750 tx=0x7f9ff400dc10 comp rx=0 tx=0).stop 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 shutdown_connections 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9fe40a3db0 0x7f9fe40a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc19a050 0x7f9ffc1a4600 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ffc106170 0x7f9ffc199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.230+0000 7fa000b51640 1 --2- 172.21.15.37:0/3754776641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ffc1010a0 0x7f9ffc199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.231+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 >> 172.21.15.37:0/3754776641 conn(0x7f9ffc077f50 msgr2=0x7f9ffc0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.231+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 shutdown_connections 2024-04-23T18:20:55.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.231+0000 7fa000b51640 1 -- 172.21.15.37:0/3754776641 wait complete. 2024-04-23T18:20:55.235 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 89.3 --await-for 42.0' 2024-04-23T18:20:55.235 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 89.3 --await-for 42.0 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 -- 172.21.15.37:0/1222743145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff100106170 msgr2=0x7ff10010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 --2- 172.21.15.37:0/1222743145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff100106170 0x7ff10010dd40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ca0 tx=0x7ff0f802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 -- 172.21.15.37:0/1222743145 shutdown_connections 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 --2- 172.21.15.37:0/1222743145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff100106170 0x7ff10010dd40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 --2- 172.21.15.37:0/1222743145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff100101a50 0x7ff100105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 --2- 172.21.15.37:0/1222743145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1001010a0 0x7ff100101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 -- 172.21.15.37:0/1222743145 >> 172.21.15.37:0/1222743145 conn(0x7ff100077f50 msgr2=0x7ff100078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 -- 172.21.15.37:0/1222743145 shutdown_connections 2024-04-23T18:20:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.360+0000 7ff1048e8640 1 -- 172.21.15.37:0/1222743145 wait complete. 2024-04-23T18:20:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 Processor -- start 2024-04-23T18:20:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 -- start start 2024-04-23T18:20:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 0x7ff100198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff100106170 0x7ff100198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10019cfc0 0x7ff10019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001110c0 con 0x7ff1001010a0 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff100110f40 con 0x7ff100106170 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff1048e8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111240 con 0x7ff10019cfc0 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.361+0000 7ff0fe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 0x7ff100198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 0x7ff100198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48088/0 (socket says 172.21.15.37:48088) 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 -- 172.21.15.37:0/701785068 learned_addr learned my addr 172.21.15.37:0/701785068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff100106170 0x7ff100198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10019cfc0 0x7ff10019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 -- 172.21.15.37:0/701785068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10019cfc0 msgr2=0x7ff10019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10019cfc0 0x7ff10019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 -- 172.21.15.37:0/701785068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff100106170 msgr2=0x7ff100198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff100106170 0x7ff100198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 -- 172.21.15.37:0/701785068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10019dab0 con 0x7ff1001010a0 2024-04-23T18:20:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 0x7ff100198700 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000e9f0 tx=0x7ff0f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:55.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0ef7fe640 1 -- 172.21.15.37:0/701785068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000cd80 con 0x7ff1001010a0 2024-04-23T18:20:55.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10019dda0 con 0x7ff1001010a0 2024-04-23T18:20:55.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0ef7fe640 1 -- 172.21.15.37:0/701785068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f00108e0 con 0x7ff1001010a0 2024-04-23T18:20:55.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff0ef7fe640 1 -- 172.21.15.37:0/701785068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0018980 con 0x7ff1001010a0 2024-04-23T18:20:55.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.362+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001a55e0 con 0x7ff1001010a0 2024-04-23T18:20:55.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:54 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.365+0000 7ff0ef7fe640 1 -- 172.21.15.37:0/701785068 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff0f0010430 con 0x7ff1001010a0 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff0ef7fe640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff0e80a3db0 0x7ff0e80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff0e80a3db0 0x7ff0e80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff0ef7fe640 1 -- 172.21.15.37:0/701785068 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff0f00d0d20 con 0x7ff1001010a0 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 Processor -- start 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 -- start start 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff0e80a3db0 0x7ff0e80a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff100100930 tx=0x7ff0f400b410 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b707480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b711b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b714480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561c5b66c370 con 0x561c5b7079c0 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561c5b6092e0 con 0x561c5b7070a0 2024-04-23T18:20:55.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.366+0000 7ff107603740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561c5b668330 con 0x561c5b712090 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b711b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b711b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48090/0 (socket says 172.21.15.37:48090) 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fdd9b640 1 -- 172.21.15.37:0/994336552 learned_addr learned my addr 172.21.15.37:0/994336552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b707480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b714480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 -- 172.21.15.37:0/994336552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 msgr2=0x561c5b714480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b714480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 -- 172.21.15.37:0/994336552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 msgr2=0x561c5b711b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b711b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 -- 172.21.15.37:0/994336552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5b44ad80 con 0x561c5b7070a0 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b711b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b707480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00ce3c0 tx=0x7ff0f009c5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1b15ca65fb9d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0ecff9640 1 -- 172.21.15.37:0/994336552 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00a0f10 con 0x561c5b7070a0 2024-04-23T18:20:55.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0ecff9640 1 -- 172.21.15.37:0/994336552 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f00a57d0 con 0x561c5b7070a0 2024-04-23T18:20:55.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.367+0000 7ff0ecff9640 1 -- 172.21.15.37:0/994336552 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00cebd0 con 0x561c5b7070a0 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 -- 172.21.15.37:0/994336552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 msgr2=0x561c5b707480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b707480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00ce3c0 tx=0x7ff0f009c5c0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 -- 172.21.15.37:0/994336552 shutdown_connections 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b714480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b711b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 --2- 172.21.15.37:0/994336552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b707480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.368+0000 7ff107603740 1 -- 172.21.15.37:0/994336552 >> 172.21.15.37:0/994336552 conn(0x561c5b7033c0 msgr2=0x561c5b7037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 -- 172.21.15.37:0/994336552 shutdown_connections 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 -- 172.21.15.37:0/994336552 wait complete. 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 -- start start 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 Processor -- start 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b70f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b70f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b70fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561c5b714dc0 con 0x561c5b7079c0 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561c5b606fb0 con 0x561c5b7070a0 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561c5b714f40 con 0x561c5b712090 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff107603740 10 client.0 _ll_register_callbacks cb 0x561c5b6069a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff0fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b70f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff0fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b70f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48096/0 (socket says 172.21.15.37:48096) 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff0fdd9b640 1 -- 172.21.15.37:0/2503501865 learned_addr learned my addr 172.21.15.37:0/2503501865 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.369+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b70f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fe59c640 1 -- 172.21.15.37:0/2503501865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 msgr2=0x561c5b70fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b70fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b70fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fe59c640 1 -- 172.21.15.37:0/2503501865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 msgr2=0x561c5b70f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b70f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fe59c640 1 -- 172.21.15.37:0/2503501865 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c5b4ab510 con 0x561c5b7070a0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b70f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b70fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b70f100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0099530 tx=0x7ff0f00ce3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5da91b209d59cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/2503501865 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00b5020 con 0x561c5b7070a0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f00a57d0 con 0x561c5b7070a0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/2503501865 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff0f00cebd0 con 0x561c5b7070a0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/2503501865 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff0f00aa050 con 0x561c5b7070a0 2024-04-23T18:20:55.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff107603740 10 client.25166 fetch_fsmap learned FSMap version 24 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.370+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561c5b33e670 con 0x561c5b7070a0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/2503501865 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f00a57d0 con 0x561c5b7070a0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/2503501865 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 24) ==== 8342+0+0 (secure 0 0 0) 0x7ff0f0018cb0 con 0x561c5b7070a0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff107603740 10 client.25166 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff107603740 10 client.25166 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff107603740 20 client.25166 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff107603740 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b7222c0 0x561c5b7246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff107603740 4 client.25166 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c5b679d90 con 0x561c5b7222c0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b7222c0 0x561c5b7246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b7222c0 0x561c5b7246b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d943eda49050f30 server_cookie=701b7efaf9eee0cb in_seq=0 out_seq=0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.371+0000 7ff0e5ffb640 10 client.25166 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/2503501865 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c5b679d90 con 0x561c5b7222c0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff0e5ffb640 10 client.25166 handle_command_reply: tid=0 2024-04-23T18:20:55.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 client.25166 shutdown 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 msgr2=0x561c5b70f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b70f100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0099530 tx=0x7ff0f00ce3f0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 shutdown_connections 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b7222c0 0x561c5b7246b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b70fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff0e5ffb640 0 client.25166 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b70f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 --2- 172.21.15.37:0/2503501865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b70f100 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 >> 172.21.15.37:0/2503501865 conn(0x561c5b7033c0 msgr2=0x561c5b7091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 shutdown_connections 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.372+0000 7ff107603740 1 -- 172.21.15.37:0/2503501865 wait complete. 2024-04-23T18:20:55.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.373+0000 7ff107603740 20 client.25166 trim_cache size 0 max 16384 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 Processor -- start 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 -- start start 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b71dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b709010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b729c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001b03e0 con 0x561c5b7079c0 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001adcb0 con 0x561c5b7070a0 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff107603740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001abb00 con 0x561c5b712090 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b709010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b709010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48104/0 (socket says 172.21.15.37:48104) 2024-04-23T18:20:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 -- 172.21.15.37:0/2784793001 learned_addr learned my addr 172.21.15.37:0/2784793001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b71dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 -- 172.21.15.37:0/2784793001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 msgr2=0x561c5b729c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b729c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b729c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 -- 172.21.15.37:0/2784793001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 msgr2=0x561c5b71dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b71dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 -- 172.21.15.37:0/2784793001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5b703f90 con 0x561c5b7079c0 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b71dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.396+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b709010 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x561c5b7106d0 tx=0x7ff0f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c285c87eb194a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff0e7fff640 1 -- 172.21.15.37:0/2784793001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f402a620 con 0x561c5b7079c0 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff0e7fff640 1 -- 172.21.15.37:0/2784793001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f402f070 con 0x561c5b7079c0 2024-04-23T18:20:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff0e7fff640 1 -- 172.21.15.37:0/2784793001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f401d610 con 0x561c5b7079c0 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 -- 172.21.15.37:0/2784793001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 msgr2=0x561c5b709010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b709010 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x561c5b7106d0 tx=0x7ff0f4013000 comp rx=0 tx=0).stop 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 -- 172.21.15.37:0/2784793001 shutdown_connections 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b729c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7079c0 0x561c5b709010 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 --2- 172.21.15.37:0/2784793001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7070a0 0x561c5b71dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 -- 172.21.15.37:0/2784793001 >> 172.21.15.37:0/2784793001 conn(0x561c5b7033c0 msgr2=0x561c5b708900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 -- 172.21.15.37:0/2784793001 shutdown_connections 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.397+0000 7ff107603740 1 -- 172.21.15.37:0/2784793001 wait complete. 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 -- start start 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 Processor -- start 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 0x561c5b71a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7079c0 0x561c5b715960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b715ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001ac170 con 0x561c5b7070a0 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001ad5e0 con 0x561c5b7079c0 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff1001ac840 con 0x561c5b712090 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff107603740 10 client.0 _ll_register_callbacks cb 0x7ff0f009c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 0x561c5b71a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 0x561c5b71a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48106/0 (socket says 172.21.15.37:48106) 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 -- 172.21.15.37:0/3125490540 learned_addr learned my addr 172.21.15.37:0/3125490540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:55.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7079c0 0x561c5b715960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b715ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 -- 172.21.15.37:0/3125490540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 msgr2=0x561c5b715ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b715ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 -- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7079c0 msgr2=0x561c5b715960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7079c0 0x561c5b715960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 -- 172.21.15.37:0/3125490540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c5b716710 con 0x561c5b7070a0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7079c0 0x561c5b715960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fed9d640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b715ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0fe59c640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 0x561c5b71a5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00a52a0 tx=0x7ff0f000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3019ad4d0b1b33e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.398+0000 7ff0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/3125490540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00b5070 con 0x561c5b7070a0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/3125490540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7ff0f00cedd0 con 0x561c5b7070a0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f402f070 con 0x561c5b7070a0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/3125490540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff0f00a57d0 con 0x561c5b7070a0 2024-04-23T18:20:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 10 client.15570 fetch_fsmap learned FSMap version 24 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561c5b788540 con 0x561c5b7070a0 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/3125490540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f402f070 con 0x561c5b7070a0 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/3125490540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8342+0+0 (secure 0 0 0) 0x7ff0f00b1020 con 0x561c5b7070a0 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 10 client.15570 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 10 client.15570 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 20 client.15570 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b788b20 0x561c5b78af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 4 client.15570 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.3, "await_for": 42.0}] 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.3, "await_for": 42.0}) -- 0x561c5b673c00 con 0x561c5b788b20 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.399+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b788b20 0x561c5b78af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.400+0000 7ff0fdd9b640 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b788b20 0x561c5b78af40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e1054f6cc487cf server_cookie=6c8cf4cf8206aa12 in_seq=0 out_seq=0 2024-04-23T18:20:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.400+0000 7ff0e5ffb640 10 client.15570 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff0e5ffb640 1 -- 172.21.15.37:0/3125490540 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x561c5b673c00 con 0x561c5b788b20 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff0e5ffb640 10 client.15570 handle_command_reply: tid=0 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 client.15570 shutdown 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 msgr2=0x561c5b71a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 0x561c5b71a5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00a52a0 tx=0x7ff0f000cf20 comp rx=0 tx=0).stop 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 shutdown_connections 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x561c5b788b20 0x561c5b78af40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561c5b712090 0x561c5b715ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff0e5ffb640 0 client.15570 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c5b7079c0 0x561c5b715960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 --2- 172.21.15.37:0/3125490540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5b7070a0 0x561c5b71a5f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 >> 172.21.15.37:0/3125490540 conn(0x561c5b7033c0 msgr2=0x561c5b70f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 shutdown_connections 2024-04-23T18:20:55.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 1 -- 172.21.15.37:0/3125490540 wait complete. 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.406+0000 7ff107603740 20 client.15570 trim_cache size 0 max 16384 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 24, 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 2, 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "5ec38c87": { 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2, 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:20:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.3, 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:20:55.402 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:20:55.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff0e80a3db0 msgr2=0x7ff0e80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff0e80a3db0 0x7ff0e80a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff100100930 tx=0x7ff0f400b410 comp rx=0 tx=0).stop 2024-04-23T18:20:55.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 msgr2=0x7ff100198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:55.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 0x7ff100198700 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000e9f0 tx=0x7ff0f000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 shutdown_connections 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff0e80a3db0 0x7ff0e80a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10019cfc0 0x7ff10019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff100106170 0x7ff100198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 --2- 172.21.15.37:0/701785068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1001010a0 0x7ff100198700 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 >> 172.21.15.37:0/701785068 conn(0x7ff100077f50 msgr2=0x7ff100069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 shutdown_connections 2024-04-23T18:20:55.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:20:55.407+0000 7ff1048e8640 1 -- 172.21.15.37:0/701785068 wait complete. 2024-04-23T18:20:55.412 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5ec38c87', quiesce took 0.6 seconds. Will release after: 29.3 2024-04-23T18:20:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:55 smithi172 ceph-mon[32456]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-23T18:20:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:55 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3754776641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:56.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:55 smithi173 ceph-mon[32906]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-23T18:20:56.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:55 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3754776641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:55 smithi037 ceph-mon[29404]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-23T18:20:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:55 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3754776641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:20:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:57 smithi172 ceph-mon[32456]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-23T18:20:58.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:57 smithi173 ceph-mon[32906]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-23T18:20:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:57 smithi037 ceph-mon[29404]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-23T18:20:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 -- 172.21.15.37:0/3882829450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e8101aa0 msgr2=0x7fc6e8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 --2- 172.21.15.37:0/3882829450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e8101aa0 0x7fc6e8105a90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009900 tx=0x7fc6d402a410 comp rx=0 tx=0).stop 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 -- 172.21.15.37:0/3882829450 shutdown_connections 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 --2- 172.21.15.37:0/3882829450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6e8106170 0x7fc6e810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 --2- 172.21.15.37:0/3882829450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e8101aa0 0x7fc6e8105a90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 --2- 172.21.15.37:0/3882829450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 -- 172.21.15.37:0/3882829450 >> 172.21.15.37:0/3882829450 conn(0x7fc6e8077fd0 msgr2=0x7fc6e80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 -- 172.21.15.37:0/3882829450 shutdown_connections 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.153+0000 7fc6eef44640 1 -- 172.21.15.37:0/3882829450 wait complete. 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 Processor -- start 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 -- start start 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6e8106170 0x7fc6e81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 0x7fc6e819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8111090 con 0x7fc6e819a710 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8110f10 con 0x7fc6e8106170 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6eef44640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8111210 con 0x7fc6e81010f0 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 0x7fc6e819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ecce0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ecce0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:32962/0 (socket says 172.21.15.37:32962) 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ecce0640 1 -- 172.21.15.37:0/618568691 learned_addr learned my addr 172.21.15.37:0/618568691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 0x7fc6e819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48134/0 (socket says 172.21.15.37:48134) 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6e8106170 0x7fc6e81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/618568691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 msgr2=0x7fc6e81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/618568691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6e8106170 msgr2=0x7fc6e81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6e8106170 0x7fc6e81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/618568691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e819b310 con 0x7fc6e819a710 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.154+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.155+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 0x7fc6e819ab90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fd0 tx=0x7fc6e000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.155+0000 7fc6e5ffb640 1 -- 172.21.15.37:0/618568691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000ce20 con 0x7fc6e819a710 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.155+0000 7fc6e5ffb640 1 -- 172.21.15.37:0/618568691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e0005020 con 0x7fc6e819a710 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.155+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e81a73f0 con 0x7fc6e819a710 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.155+0000 7fc6e5ffb640 1 -- 172.21.15.37:0/618568691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0010690 con 0x7fc6e819a710 2024-04-23T18:20:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.155+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e81a77a0 con 0x7fc6e819a710 2024-04-23T18:20:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.157+0000 7fc6e5ffb640 1 -- 172.21.15.37:0/618568691 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc6e00054d0 con 0x7fc6e819a710 2024-04-23T18:20:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.158+0000 7fc6e5ffb640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3ee0 0x7fc6d80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.158+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3ee0 0x7fc6d80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.158+0000 7fc6e5ffb640 1 -- 172.21.15.37:0/618568691 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc6e00d0fb0 con 0x7fc6e819a710 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.158+0000 7fc6f1c69740 1 Processor -- start 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 -- start start 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3ee0 0x7fc6d80a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005fd0 tx=0x7fc6dc00b480 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 0x557f337eca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337f7800 0x557f337f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f33751970 con 0x557f337ec620 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f336ee4a0 con 0x557f337ecf40 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557f3374d930 con 0x557f337f7800 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 0x557f337eca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 0x557f337eca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48138/0 (socket says 172.21.15.37:48138) 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 -- 172.21.15.37:0/2009876730 learned_addr learned my addr 172.21.15.37:0/2009876730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337f7800 0x557f337f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 -- 172.21.15.37:0/2009876730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337f7800 msgr2=0x557f337f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337f7800 0x557f337f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 -- 172.21.15.37:0/2009876730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 msgr2=0x557f337f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 -- 172.21.15.37:0/2009876730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f33464210 con 0x557f337ec620 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.159+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 0x557f337eca00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc011990 tx=0x7fc6dc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e681c4270a7217 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.160+0000 7fc6d2ffd640 1 -- 172.21.15.37:0/2009876730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc01cac0 con 0x557f337ec620 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.160+0000 7fc6d2ffd640 1 -- 172.21.15.37:0/2009876730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc0054d0 con 0x557f337ec620 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.160+0000 7fc6d2ffd640 1 -- 172.21.15.37:0/2009876730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004d10 con 0x557f337ec620 2024-04-23T18:20:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.160+0000 7fc6f1c69740 1 -- 172.21.15.37:0/2009876730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 msgr2=0x557f337eca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.160+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 0x557f337eca00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc011990 tx=0x7fc6dc013000 comp rx=0 tx=0).stop 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- 172.21.15.37:0/2009876730 shutdown_connections 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337f7800 0x557f337f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/2009876730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ec620 0x557f337eca00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- 172.21.15.37:0/2009876730 >> 172.21.15.37:0/2009876730 conn(0x557f337e8b30 msgr2=0x557f337e8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- 172.21.15.37:0/2009876730 shutdown_connections 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- 172.21.15.37:0/2009876730 wait complete. 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- start start 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 Processor -- start 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ec620 0x557f337f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ecf40 0x557f337f4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 0x557f337f4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f337fa430 con 0x557f337f7800 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f336ec0b0 con 0x557f337ec620 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557f337fa5b0 con 0x557f337ecf40 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6f1c69740 10 client.0 _ll_register_callbacks cb 0x557f336d9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.161+0000 7fc6ed4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 0x557f337f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 0x557f337f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48142/0 (socket says 172.21.15.37:48142) 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/4031114232 learned_addr learned my addr 172.21.15.37:0/4031114232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ecf40 0x557f337f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ec620 0x557f337f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/4031114232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ecf40 msgr2=0x557f337f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ecf40 0x557f337f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ec620 msgr2=0x557f337f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ec620 0x557f337f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/4031114232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f33530620 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ec620 0x557f337f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 0x557f337f4960 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc6e009cd20 tx=0x7fc6e009d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98aa2a0f34e20a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6c7fff640 1 -- 172.21.15.37:0/4031114232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00b5070 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6c7fff640 1 -- 172.21.15.37:0/4031114232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc6e00b6260 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6dc0054d0 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6c7fff640 1 -- 172.21.15.37:0/4031114232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6e00a5a00 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6f1c69740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f33806310 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.162+0000 7fc6c7fff640 1 -- 172.21.15.37:0/4031114232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc0054d0 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6c7fff640 1 -- 172.21.15.37:0/4031114232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc6e00bd0c0 con 0x557f337f7800 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6f1c69740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:20:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6f1c69740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6f1c69740 20 client.15582 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f338077d0 0x557f33809c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6f1c69740 4 client.15582 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f3375f390 con 0x557f338077d0 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.163+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f338077d0 0x557f33809c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.164+0000 7fc6c7fff640 10 client.15582 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.164+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f338077d0 0x557f33809c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a7522c436d4acf server_cookie=9d7c49441f8a62 in_seq=0 out_seq=0 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.164+0000 7fc6c7fff640 1 -- 172.21.15.37:0/4031114232 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f3375f390 con 0x557f338077d0 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.164+0000 7fc6c7fff640 10 client.15582 handle_command_reply: tid=0 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.164+0000 7fc6f1c69740 1 client.15582 shutdown 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.164+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 msgr2=0x557f337f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 0x557f337f4960 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc6e009cd20 tx=0x7fc6e009d5d0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 shutdown_connections 2024-04-23T18:20:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f338077d0 0x557f33809c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6c7fff640 0 client.15582 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337f7800 0x557f337f4960 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ecf40 0x557f337f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/4031114232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ec620 0x557f337f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 >> 172.21.15.37:0/4031114232 conn(0x557f337e8b30 msgr2=0x557f337ee8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 shutdown_connections 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 1 -- 172.21.15.37:0/4031114232 wait complete. 2024-04-23T18:20:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.165+0000 7fc6f1c69740 20 client.15582 trim_cache size 0 max 16384 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 Processor -- start 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 -- start start 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337e8b30 0x557f338032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f3380f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 0x557f3380fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81b25a0 con 0x557f337ecf40 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81afe70 con 0x557f337e8b30 2024-04-23T18:20:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81adcc0 con 0x557f337ec620 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 0x557f3380fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 0x557f3380fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48152/0 (socket says 172.21.15.37:48152) 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/3787485510 learned_addr learned my addr 172.21.15.37:0/3787485510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337e8b30 0x557f338032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f3380f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/3787485510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 msgr2=0x557f3380f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f3380f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/3787485510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337e8b30 msgr2=0x557f338032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337e8b30 0x557f338032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ed4e1640 1 -- 172.21.15.37:0/3787485510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3380a280 con 0x557f337ecf40 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.180+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337e8b30 0x557f338032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.181+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 0x557f3380fa50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00a54d0 tx=0x7fc6e00053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccfac47d934fda server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.181+0000 7fc6d27fc640 1 -- 172.21.15.37:0/3787485510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00b5070 con 0x557f337ecf40 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.181+0000 7fc6d27fc640 1 -- 172.21.15.37:0/3787485510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e00b6260 con 0x557f337ecf40 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.181+0000 7fc6d27fc640 1 -- 172.21.15.37:0/3787485510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e007be10 con 0x557f337ecf40 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- 172.21.15.37:0/3787485510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 msgr2=0x557f3380fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 0x557f3380fa50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00a54d0 tx=0x7fc6e00053e0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- 172.21.15.37:0/3787485510 shutdown_connections 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337ecf40 0x557f3380fa50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f3380f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/3787485510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337e8b30 0x557f338032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- 172.21.15.37:0/3787485510 >> 172.21.15.37:0/3787485510 conn(0x557f337f4ac0 msgr2=0x557f337e7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- 172.21.15.37:0/3787485510 shutdown_connections 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- 172.21.15.37:0/3787485510 wait complete. 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- start start 2024-04-23T18:20:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 Processor -- start 2024-04-23T18:20:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 0x557f337ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f337fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ae330 con 0x557f337e8b30 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81af7a0 con 0x557f337ecf40 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.182+0000 7fc6f1c69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81aea00 con 0x557f337ec620 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6f1c69740 10 client.0 _ll_register_callbacks cb 0x7fc6e009c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 0x557f337ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 0x557f337ffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48162/0 (socket says 172.21.15.37:48162) 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 -- 172.21.15.37:0/1773292739 learned_addr learned my addr 172.21.15.37:0/1773292739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f337fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 -- 172.21.15.37:0/1773292739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 msgr2=0x557f337fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f337fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 -- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 msgr2=0x557f337fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ed4e1640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 -- 172.21.15.37:0/1773292739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f337fd910 con 0x557f337e8b30 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6ecce0640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 0x557f337ffa00 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc011960 tx=0x7fc6dc010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f6ae0f6261fb19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6c7fff640 1 -- 172.21.15.37:0/1773292739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc03b060 con 0x557f337e8b30 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6c7fff640 1 -- 172.21.15.37:0/1773292739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc6dc01cc50 con 0x557f337e8b30 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.183+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e00b6260 con 0x557f337e8b30 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6c7fff640 1 -- 172.21.15.37:0/1773292739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6dc01ce70 con 0x557f337e8b30 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f3386db40 con 0x557f337e8b30 2024-04-23T18:20:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6c7fff640 1 -- 172.21.15.37:0/1773292739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e00b6260 con 0x557f337e8b30 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6c7fff640 1 -- 172.21.15.37:0/1773292739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc6dc0325e0 con 0x557f337e8b30 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 20 client.15588 populate_metadata read hostname 'smithi037' 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f337f7800 0x557f33870170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 4 client.15588 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f33759200 con 0x557f337f7800 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.184+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f337f7800 0x557f33870170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.185+0000 7fc6e7fff640 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f337f7800 0x557f33870170 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5783a18da8539b server_cookie=34370f03c49a1c04 in_seq=0 out_seq=0 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.185+0000 7fc6c7fff640 10 client.15588 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.188+0000 7fc6c7fff640 1 -- 172.21.15.37:0/1773292739 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f33759200 con 0x557f337f7800 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.188+0000 7fc6c7fff640 10 client.15588 handle_command_reply: tid=0 2024-04-23T18:20:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.188+0000 7fc6f1c69740 1 client.15588 shutdown 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 msgr2=0x557f337ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 0x557f337ffa00 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc011960 tx=0x7fc6dc010e60 comp rx=0 tx=0).stop 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 shutdown_connections 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557f337f7800 0x557f33870170 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f337ecf40 0x557f337fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6c7fff640 0 client.15588 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557f337ec620 0x557f337fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 --2- 172.21.15.37:0/1773292739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f337e8b30 0x557f337ffa00 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 >> 172.21.15.37:0/1773292739 conn(0x557f337f4ac0 msgr2=0x557f337e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 shutdown_connections 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 1 -- 172.21.15.37:0/1773292739 wait complete. 2024-04-23T18:20:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6f1c69740 20 client.15588 trim_cache size 0 max 16384 2024-04-23T18:20:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:20:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:20:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:20:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3ee0 msgr2=0x7fc6d80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6eef44640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3ee0 0x7fc6d80a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005fd0 tx=0x7fc6dc00b480 comp rx=0 tx=0).stop 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 msgr2=0x7fc6e819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.189+0000 7fc6eef44640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 0x7fc6e819ab90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fd0 tx=0x7fc6e000ef10 comp rx=0 tx=0).stop 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 shutdown_connections 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3ee0 0x7fc6d80a63a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6e819a710 0x7fc6e819ab90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6e8106170 0x7fc6e81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 --2- 172.21.15.37:0/618568691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e81010f0 0x7fc6e81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 >> 172.21.15.37:0/618568691 conn(0x7fc6e8077fd0 msgr2=0x7fc6e80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 shutdown_connections 2024-04-23T18:20:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:20:59.190+0000 7fc6eef44640 1 -- 172.21.15.37:0/618568691 wait complete. 2024-04-23T18:21:00.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:59 smithi172 ceph-mon[32456]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-23T18:21:00.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:20:59 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:21:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:59 smithi173 ceph-mon[32906]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-23T18:21:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:20:59 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:21:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:59 smithi037 ceph-mon[29404]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-23T18:21:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:20:59 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:21:00.370 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:21:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:21:00.059+0000 7fbafa2bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:21:00.370 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:21:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:21:00.059+0000 7fbafa2bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:21:00.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:21:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:21:00.059+0000 7fbafa2bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:21:00.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:21:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:21:00.059+0000 7fbafa2bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:21:00.371 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:21:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:21:00.059+0000 7fbafa2bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:21:02.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:01 smithi172 ceph-mon[32456]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T18:21:02.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:01 smithi172 ceph-mon[32456]: mgrmap e25: x(active, since 30s), standbys: z, y 2024-04-23T18:21:02.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:01 smithi173 ceph-mon[32906]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T18:21:02.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:01 smithi173 ceph-mon[32906]: mgrmap e25: x(active, since 30s), standbys: z, y 2024-04-23T18:21:02.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:01 smithi037 ceph-mon[29404]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T18:21:02.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:01 smithi037 ceph-mon[29404]: mgrmap e25: x(active, since 30s), standbys: z, y 2024-04-23T18:21:04.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:03 smithi172 ceph-mon[32456]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:21:04.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:03 smithi173 ceph-mon[32906]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:21:04.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:03 smithi037 ceph-mon[29404]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-23T18:21:06.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:05 smithi172 ceph-mon[32456]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:21:06.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:05 smithi173 ceph-mon[32906]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:21:06.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:05 smithi037 ceph-mon[29404]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:21:08.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:07 smithi172 ceph-mon[32456]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T18:21:08.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:07 smithi173 ceph-mon[32906]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T18:21:08.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:07 smithi037 ceph-mon[29404]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T18:21:10.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:09 smithi172 ceph-mon[32456]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T18:21:10.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:09 smithi173 ceph-mon[32906]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T18:21:10.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:09 smithi037 ceph-mon[29404]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T18:21:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 -- 172.21.15.37:0/2865595626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 msgr2=0x7fde3c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 --2- 172.21.15.37:0/2865595626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c10dd40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fde24009900 tx=0x7fde2402a410 comp rx=0 tx=0).stop 2024-04-23T18:21:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 -- 172.21.15.37:0/2865595626 shutdown_connections 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 --2- 172.21.15.37:0/2865595626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c10dd40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 --2- 172.21.15.37:0/2865595626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde3c101aa0 0x7fde3c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 --2- 172.21.15.37:0/2865595626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde3c1010f0 0x7fde3c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 -- 172.21.15.37:0/2865595626 >> 172.21.15.37:0/2865595626 conn(0x7fde3c077fd0 msgr2=0x7fde3c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 -- 172.21.15.37:0/2865595626 shutdown_connections 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.328+0000 7fde430cf640 1 -- 172.21.15.37:0/2865595626 wait complete. 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 Processor -- start 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 -- start start 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde3c1010f0 0x7fde3c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde3c197d90 0x7fde3c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c111130 con 0x7fde3c106170 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c110fb0 con 0x7fde3c197d90 2024-04-23T18:21:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde430cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1112b0 con 0x7fde3c1010f0 2024-04-23T18:21:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34682/0 (socket says 172.21.15.37:34682) 2024-04-23T18:21:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde3bfff640 1 -- 172.21.15.37:0/4241646745 learned_addr learned my addr 172.21.15.37:0/4241646745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde40e6b640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde3c1010f0 0x7fde3c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.329+0000 7fde4166c640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde3c197d90 0x7fde3c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde3bfff640 1 -- 172.21.15.37:0/4241646745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde3c1010f0 msgr2=0x7fde3c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde3bfff640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde3c1010f0 0x7fde3c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde3bfff640 1 -- 172.21.15.37:0/4241646745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde3c197d90 msgr2=0x7fde3c19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde3bfff640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde3c197d90 0x7fde3c19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde3bfff640 1 -- 172.21.15.37:0/4241646745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c19cb10 con 0x7fde3c106170 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde3bfff640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c197800 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00d740 tx=0x7fde2c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde39ffb640 1 -- 172.21.15.37:0/4241646745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c017070 con 0x7fde3c106170 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde39ffb640 1 -- 172.21.15.37:0/4241646745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2c00de50 con 0x7fde3c106170 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c19ce00 con 0x7fde3c106170 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.330+0000 7fde39ffb640 1 -- 172.21.15.37:0/4241646745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c004ce0 con 0x7fde3c106170 2024-04-23T18:21:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.331+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c19d350 con 0x7fde3c106170 2024-04-23T18:21:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.334+0000 7fde39ffb640 1 -- 172.21.15.37:0/4241646745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fde2c004e80 con 0x7fde3c106170 2024-04-23T18:21:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.334+0000 7fde45de9740 1 Processor -- start 2024-04-23T18:21:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.334+0000 7fde45de9740 1 -- start start 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde45de9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850ad9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde45de9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde45de9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850ae6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde45de9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563850a3e970 con 0x563850ad9620 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde45de9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5638509db4a0 con 0x563850ad9f40 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde45de9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563850a3a930 con 0x563850ae4800 2024-04-23T18:21:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde39ffb640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fde300a3f60 0x7fde300a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850ad9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850ad9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34694/0 (socket says 172.21.15.37:34694) 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 -- 172.21.15.37:0/1269014452 learned_addr learned my addr 172.21.15.37:0/1269014452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde39ffb640 1 -- 172.21.15.37:0/4241646745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fde2c0cffd0 con 0x7fde3c106170 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fde300a3f60 0x7fde300a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde3bfff640 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850ae6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 -- 172.21.15.37:0/1269014452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 msgr2=0x563850ae6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850ae6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 -- 172.21.15.37:0/1269014452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 msgr2=0x563850ae42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde40e6b640 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde4166c640 1 -- 172.21.15.37:0/1269014452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563850751210 con 0x563850ad9620 2024-04-23T18:21:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.335+0000 7fde40e6b640 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.336+0000 7fde4166c640 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850ad9a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fde2402aa00 tx=0x7fde24003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b6a9722b85ba9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.336+0000 7fde4166c640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fde300a3f60 0x7fde300a6420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde24009900 tx=0x7fde24040000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.336+0000 7fde2affd640 1 -- 172.21.15.37:0/1269014452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24035040 con 0x563850ad9620 2024-04-23T18:21:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.336+0000 7fde2affd640 1 -- 172.21.15.37:0/1269014452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2404d070 con 0x563850ad9620 2024-04-23T18:21:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.336+0000 7fde2affd640 1 -- 172.21.15.37:0/1269014452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24048430 con 0x563850ad9620 2024-04-23T18:21:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 -- 172.21.15.37:0/1269014452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 msgr2=0x563850ad9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850ad9a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fde2402aa00 tx=0x7fde24003910 comp rx=0 tx=0).stop 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 -- 172.21.15.37:0/1269014452 shutdown_connections 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850ae6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 --2- 172.21.15.37:0/1269014452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850ad9a00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 -- 172.21.15.37:0/1269014452 >> 172.21.15.37:0/1269014452 conn(0x563850ad5b30 msgr2=0x563850ad5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 -- 172.21.15.37:0/1269014452 shutdown_connections 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.337+0000 7fde45de9740 1 -- 172.21.15.37:0/1269014452 wait complete. 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.338+0000 7fde45de9740 1 -- start start 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.338+0000 7fde45de9740 1 Processor -- start 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde45de9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9620 0x563850ae4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde45de9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ad9f40 0x563850ae1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde45de9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 0x563850ae1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde45de9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563850ae73d0 con 0x563850ae4800 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde45de9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5638509d90b0 con 0x563850ad9620 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde45de9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563850ae7550 con 0x563850ad9f40 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde40e6b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9620 0x563850ae4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde4166c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 0x563850ae1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde4166c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 0x563850ae1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34696/0 (socket says 172.21.15.37:34696) 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde4166c640 1 -- 172.21.15.37:0/778254279 learned_addr learned my addr 172.21.15.37:0/778254279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde3bfff640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ad9f40 0x563850ae1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde40e6b640 1 -- 172.21.15.37:0/778254279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ad9f40 msgr2=0x563850ae1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde40e6b640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ad9f40 0x563850ae1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde40e6b640 1 -- 172.21.15.37:0/778254279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 msgr2=0x563850ae1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde40e6b640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 0x563850ae1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde40e6b640 1 -- 172.21.15.37:0/778254279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56385081d620 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.339+0000 7fde4166c640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 0x563850ae1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde45de9740 10 client.0 _ll_register_callbacks cb 0x5638509c6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2404d070 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde40e6b640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9620 0x563850ae4290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde34005fd0 tx=0x7fde34012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=464a7a7cee36add5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde1bfff640 10 client.25193 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde1bfff640 1 -- 172.21.15.37:0/778254279 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34010e50 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde1bfff640 1 -- 172.21.15.37:0/778254279 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fde34003970 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde1bfff640 1 -- 172.21.15.37:0/778254279 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde34003bd0 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde45de9740 10 client.25193 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=25}) -- 0x563850af31b0 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.340+0000 7fde1bfff640 1 -- 172.21.15.37:0/778254279 <== mon.2 v2:172.21.15.172:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde34004050 con 0x563850ad9620 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde45de9740 10 client.25193 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde45de9740 10 client.25193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde45de9740 20 client.25193 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde45de9740 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850af4670 0x563850af6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde45de9740 4 client.25193 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563850a4c390 con 0x563850af4670 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde3bfff640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850af4670 0x563850af6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde1bfff640 10 client.25193 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.341+0000 7fde3bfff640 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850af4670 0x563850af6b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ce6e9246436271 server_cookie=e208f658ecaa3912 in_seq=0 out_seq=0 2024-04-23T18:21:12.339 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:11 smithi172 ceph-mon[32456]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde1bfff640 1 -- 172.21.15.37:0/778254279 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563850a4c390 con 0x563850af4670 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde1bfff640 10 client.25193 handle_command_reply: tid=0 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 client.25193 shutdown 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9620 msgr2=0x563850ae4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9620 0x563850ae4290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde34005fd0 tx=0x7fde34012e70 comp rx=0 tx=0).stop 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 shutdown_connections 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850af4670 0x563850af6b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ae4800 0x563850ae1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde1bfff640 0 client.25193 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ad9f40 0x563850ae1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 --2- 172.21.15.37:0/778254279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9620 0x563850ae4290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.342+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 >> 172.21.15.37:0/778254279 conn(0x563850ad5b30 msgr2=0x563850adb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.343+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 shutdown_connections 2024-04-23T18:21:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.343+0000 7fde45de9740 1 -- 172.21.15.37:0/778254279 wait complete. 2024-04-23T18:21:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.343+0000 7fde45de9740 20 client.25193 trim_cache size 0 max 16384 2024-04-23T18:21:12.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:11 smithi173 ceph-mon[32906]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.372+0000 7fde45de9740 1 Processor -- start 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.372+0000 7fde45de9740 1 -- start start 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde45de9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850af0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde45de9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde45de9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850afc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde45de9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b6840 con 0x563850ad9620 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde45de9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b4110 con 0x563850ad9f40 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde45de9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b1f60 con 0x563850ae4800 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850af0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850af0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34708/0 (socket says 172.21.15.37:34708) 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 -- 172.21.15.37:0/881733368 learned_addr learned my addr 172.21.15.37:0/881733368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde3bfff640 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde4166c640 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850afc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 -- 172.21.15.37:0/881733368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 msgr2=0x563850afc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850afc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 -- 172.21.15.37:0/881733368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 msgr2=0x563850ae3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde40e6b640 1 -- 172.21.15.37:0/881733368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563850ae6f60 con 0x563850ad9620 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.373+0000 7fde3bfff640 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.374+0000 7fde40e6b640 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850af0290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fde340108a0 tx=0x7fde34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dde7a4ddaf11ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.374+0000 7fde2a7fc640 1 -- 172.21.15.37:0/881733368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34033070 con 0x563850ad9620 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.374+0000 7fde2a7fc640 1 -- 172.21.15.37:0/881733368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2404d070 con 0x563850ad9620 2024-04-23T18:21:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.374+0000 7fde2a7fc640 1 -- 172.21.15.37:0/881733368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde340041f0 con 0x563850ad9620 2024-04-23T18:21:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 -- 172.21.15.37:0/881733368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 msgr2=0x563850af0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850af0290 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fde340108a0 tx=0x7fde34015000 comp rx=0 tx=0).stop 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 -- 172.21.15.37:0/881733368 shutdown_connections 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ae4800 0x563850afc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ad9f40 0x563850ae3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 --2- 172.21.15.37:0/881733368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9620 0x563850af0290 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 -- 172.21.15.37:0/881733368 >> 172.21.15.37:0/881733368 conn(0x563850ad5b30 msgr2=0x563850ae1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 -- 172.21.15.37:0/881733368 shutdown_connections 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 -- 172.21.15.37:0/881733368 wait complete. 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 -- start start 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.375+0000 7fde45de9740 1 Processor -- start 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 0x563850aecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ae4800 0x563850adde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ade3b0 0x563850ade830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b25d0 con 0x563850ad9f40 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b3a40 con 0x563850ae4800 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b2ca0 con 0x563850ade3b0 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde45de9740 10 client.0 _ll_register_callbacks cb 0x7fde34012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde40e6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 0x563850aecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde40e6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 0x563850aecbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34720/0 (socket says 172.21.15.37:34720) 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde40e6b640 1 -- 172.21.15.37:0/3950779765 learned_addr learned my addr 172.21.15.37:0/3950779765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde4166c640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ade3b0 0x563850ade830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.376+0000 7fde3bfff640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ae4800 0x563850adde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde4166c640 1 -- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ae4800 msgr2=0x563850adde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde4166c640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ae4800 0x563850adde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde4166c640 1 -- 172.21.15.37:0/3950779765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 msgr2=0x563850aecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde4166c640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 0x563850aecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde4166c640 1 -- 172.21.15.37:0/3950779765 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563850b59df0 con 0x563850ade3b0 2024-04-23T18:21:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde40e6b640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 0x563850aecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde4166c640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ade3b0 0x563850ade830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x563850a84560 tx=0x7fde24031a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b127ef00a73f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:21:12.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:11 smithi037 ceph-mon[29404]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde1bfff640 1 -- 172.21.15.37:0/3950779765 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24035040 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde1bfff640 1 -- 172.21.15.37:0/3950779765 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fde24005040 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.377+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2404d070 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde1bfff640 1 -- 172.21.15.37:0/3950779765 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde24004460 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 10 client.25186 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563850b5b040 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde1bfff640 1 -- 172.21.15.37:0/3950779765 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde240046c0 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde1bfff640 1 -- 172.21.15.37:0/3950779765 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde24047070 con 0x563850ade3b0 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 10 client.25186 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 10 client.25186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 20 client.25186 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850b5b320 0x563850b5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 4 client.25186 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:21:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fde2404d070 con 0x563850b5b320 2024-04-23T18:21:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.378+0000 7fde40e6b640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850b5b320 0x563850b5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.379+0000 7fde40e6b640 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850b5b320 0x563850b5d710 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83771098edeb27f9 server_cookie=271d10802dd8f06b in_seq=0 out_seq=0 2024-04-23T18:21:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.379+0000 7fde1bfff640 10 client.25186 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.386+0000 7fde1bfff640 1 -- 172.21.15.37:0/3950779765 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fde2404d070 con 0x563850b5b320 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.386+0000 7fde1bfff640 10 client.25186 handle_command_reply: tid=0 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.386+0000 7fde45de9740 1 client.25186 shutdown 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.386+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ade3b0 msgr2=0x563850ade830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.386+0000 7fde45de9740 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ade3b0 0x563850ade830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x563850a84560 tx=0x7fde24031a80 comp rx=0 tx=0).stop 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 shutdown_connections 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563850b5b320 0x563850b5d710 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563850ade3b0 0x563850ade830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563850ae4800 0x563850adde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 --2- 172.21.15.37:0/3950779765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563850ad9f40 0x563850aecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 >> 172.21.15.37:0/3950779765 conn(0x563850ad5b30 msgr2=0x563850adafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde1bfff640 0 client.25186 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 shutdown_connections 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 1 -- 172.21.15.37:0/3950779765 wait complete. 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.387+0000 7fde45de9740 20 client.25186 trim_cache size 0 max 16384 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:21:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fde300a3f60 msgr2=0x7fde300a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fde300a3f60 0x7fde300a6420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde24009900 tx=0x7fde24040000 comp rx=0 tx=0).stop 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 msgr2=0x7fde3c197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c197800 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00d740 tx=0x7fde2c00dc00 comp rx=0 tx=0).stop 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 shutdown_connections 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fde300a3f60 0x7fde300a6420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde3c197d90 0x7fde3c19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c106170 0x7fde3c197800 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 --2- 172.21.15.37:0/4241646745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde3c1010f0 0x7fde3c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 >> 172.21.15.37:0/4241646745 conn(0x7fde3c077fd0 msgr2=0x7fde3c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 shutdown_connections 2024-04-23T18:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:12.388+0000 7fde430cf640 1 -- 172.21.15.37:0/4241646745 wait complete. 2024-04-23T18:21:14.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:13 smithi037 ceph-mon[29404]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-23T18:21:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:13 smithi172 ceph-mon[32456]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-23T18:21:14.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:13 smithi173 ceph-mon[32906]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-23T18:21:16.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:15 smithi172 ceph-mon[32456]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-23T18:21:16.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:15 smithi173 ceph-mon[32906]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-23T18:21:16.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:15 smithi037 ceph-mon[29404]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-23T18:21:18.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:17 smithi172 ceph-mon[32456]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:21:18.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:17 smithi173 ceph-mon[32906]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:21:18.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:17 smithi037 ceph-mon[29404]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:21:20.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:19 smithi172 ceph-mon[32456]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T18:21:20.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:19 smithi173 ceph-mon[32906]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T18:21:20.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:19 smithi037 ceph-mon[29404]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T18:21:22.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:21 smithi172 ceph-mon[32456]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-23T18:21:22.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:21 smithi173 ceph-mon[32906]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-23T18:21:22.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:21 smithi037 ceph-mon[29404]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-23T18:21:24.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:23 smithi172 ceph-mon[32456]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-23T18:21:24.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:23 smithi173 ceph-mon[32906]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-23T18:21:24.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:23 smithi037 ceph-mon[29404]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-23T18:21:24.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 -- 172.21.15.37:0/3482499315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e8106170 msgr2=0x7f39e810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/3482499315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e8106170 0x7f39e810dd40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009ca0 tx=0x7f39e002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 -- 172.21.15.37:0/3482499315 shutdown_connections 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/3482499315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e8106170 0x7f39e810dd40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/3482499315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e8101a50 0x7f39e8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/3482499315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e81010a0 0x7f39e8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.834+0000 7f39ed8a7640 1 -- 172.21.15.37:0/3482499315 >> 172.21.15.37:0/3482499315 conn(0x7f39e8077f50 msgr2=0x7f39e8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.835+0000 7f39ed8a7640 1 -- 172.21.15.37:0/3482499315 shutdown_connections 2024-04-23T18:21:24.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.835+0000 7f39ed8a7640 1 -- 172.21.15.37:0/3482499315 wait complete. 2024-04-23T18:21:24.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.835+0000 7f39ed8a7640 1 Processor -- start 2024-04-23T18:21:24.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.835+0000 7f39ed8a7640 1 -- start start 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39ed8a7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e81010a0 0x7f39e8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39ed8a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 0x7f39e8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39ed8a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819cfe0 0x7f39e819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39ed8a7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81110e0 con 0x7f39e819cfe0 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39ed8a7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8110f60 con 0x7f39e8106170 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39ed8a7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8111260 con 0x7f39e81010a0 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 0x7f39e8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 0x7f39e8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:41296/0 (socket says 172.21.15.37:41296) 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 -- 172.21.15.37:0/4052932241 learned_addr learned my addr 172.21.15.37:0/4052932241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e77fe640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819cfe0 0x7f39e819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e6ffd640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e81010a0 0x7f39e8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 -- 172.21.15.37:0/4052932241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e81010a0 msgr2=0x7f39e8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e81010a0 0x7f39e8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 -- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819cfe0 msgr2=0x7f39e819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819cfe0 0x7f39e819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e67fc640 1 -- 172.21.15.37:0/4052932241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e819dad0 con 0x7f39e8106170 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.836+0000 7f39e6ffd640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e81010a0 0x7f39e8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.837+0000 7f39e67fc640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 0x7f39e8198cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e9f0 tx=0x7f39dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.837+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00ccc0 con 0x7f39e8106170 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.837+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39dc0054d0 con 0x7f39e8106170 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.837+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc010430 con 0x7f39e8106170 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.837+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e819ddc0 con 0x7f39e8106170 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.837+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e81a5560 con 0x7f39e8106170 2024-04-23T18:21:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.838+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e8068d70 con 0x7f39e8106170 2024-04-23T18:21:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.840+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f39dc0046c0 con 0x7f39e8106170 2024-04-23T18:21:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.840+0000 7f39ec8a5640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f39d00a3e40 0x7f39d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.841+0000 7f39e6ffd640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f39d00a3e40 0x7f39d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.841+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f39dc0d0dd0 con 0x7f39e8106170 2024-04-23T18:21:24.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.841+0000 7f39e6ffd640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f39d00a3e40 0x7f39d00a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009cd0 tx=0x7f39d8005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:24.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:24.846+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39dc09d230 con 0x7f39e8106170 2024-04-23T18:21:25.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.074+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e8101b20 con 0x7f39e8106170 2024-04-23T18:21:25.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.075+0000 7f39ec8a5640 1 -- 172.21.15.37:0/4052932241 <== mon.2 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11878 (secure 0 0 0) 0x7f39dc09cb30 con 0x7f39e8106170 2024-04-23T18:21:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:21:25.069 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:58.538288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:21:25.069 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f39d00a3e40 msgr2=0x7f39d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f39d00a3e40 0x7f39d00a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009cd0 tx=0x7f39d8005f90 comp rx=0 tx=0).stop 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 msgr2=0x7f39e8198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 0x7f39e8198cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e9f0 tx=0x7f39dc00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 shutdown_connections 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f39d00a3e40 0x7f39d00a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819cfe0 0x7f39e819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39e8106170 0x7f39e8198cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 --2- 172.21.15.37:0/4052932241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e81010a0 0x7f39e8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.077+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 >> 172.21.15.37:0/4052932241 conn(0x7f39e8077f50 msgr2=0x7f39e80ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.078+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 shutdown_connections 2024-04-23T18:21:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.078+0000 7f39ed8a7640 1 -- 172.21.15.37:0/4052932241 wait complete. 2024-04-23T18:21:25.086 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 5ec38c87 --release --await' 2024-04-23T18:21:25.086 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id 5ec38c87 --release --await 2024-04-23T18:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 -- 172.21.15.37:0/298698648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0280103af0 msgr2=0x7f028010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 --2- 172.21.15.37:0/298698648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0280103af0 0x7f028010bfd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0274009860 tx=0x7f027402a460 comp rx=0 tx=0).stop 2024-04-23T18:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 -- 172.21.15.37:0/298698648 shutdown_connections 2024-04-23T18:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 --2- 172.21.15.37:0/298698648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028010c510 0x7f028010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 --2- 172.21.15.37:0/298698648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0280103af0 0x7f028010bfd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 --2- 172.21.15.37:0/298698648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02801031d0 0x7f02801035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 -- 172.21.15.37:0/298698648 >> 172.21.15.37:0/298698648 conn(0x7f0280077f50 msgr2=0x7f0280078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 -- 172.21.15.37:0/298698648 shutdown_connections 2024-04-23T18:21:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.212+0000 7f0285aa0640 1 -- 172.21.15.37:0/298698648 wait complete. 2024-04-23T18:21:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 Processor -- start 2024-04-23T18:21:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 -- start start 2024-04-23T18:21:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 0x7f0280198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0280103af0 0x7f0280198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028010c510 0x7f028019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f02801121f0 con 0x7f028010c510 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0280112070 con 0x7f02801031d0 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f0285aa0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0280112370 con 0x7f0280103af0 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.213+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 0x7f0280198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 0x7f0280198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:41316/0 (socket says 172.21.15.37:41316) 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 -- 172.21.15.37:0/1347822780 learned_addr learned my addr 172.21.15.37:0/1347822780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 -- 172.21.15.37:0/1347822780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0280103af0 msgr2=0x7f0280198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0280103af0 0x7f0280198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 -- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028010c510 msgr2=0x7f028019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027ffff640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028010c510 0x7f028019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028010c510 0x7f028019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 -- 172.21.15.37:0/1347822780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028019d740 con 0x7f02801031d0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027ffff640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028010c510 0x7f028019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.214+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 0x7f0280198750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f027000e9c0 tx=0x7f027000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.215+0000 7f027cff9640 1 -- 172.21.15.37:0/1347822780 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027000cde0 con 0x7f02801031d0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.215+0000 7f027cff9640 1 -- 172.21.15.37:0/1347822780 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02700035f0 con 0x7f02801031d0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.215+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028019da30 con 0x7f02801031d0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.215+0000 7f027cff9640 1 -- 172.21.15.37:0/1347822780 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270010690 con 0x7f02801031d0 2024-04-23T18:21:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.215+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0280100d50 con 0x7f02801031d0 2024-04-23T18:21:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.217+0000 7f027cff9640 1 -- 172.21.15.37:0/1347822780 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f0270004000 con 0x7f02801031d0 2024-04-23T18:21:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.217+0000 7f027cff9640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02680a3ec0 0x7f02680a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f027cff9640 1 -- 172.21.15.37:0/1347822780 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f02700d0e70 con 0x7f02801031d0 2024-04-23T18:21:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f027effd640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02680a3ec0 0x7f02680a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 Processor -- start 2024-04-23T18:21:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 -- start start 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f027effd640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02680a3ec0 0x7f02680a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f028010e360 tx=0x7f0274005c90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bb6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bc1480 0x563737bc3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563737b1b6d0 con 0x563737bb6db0 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563737ab81a0 con 0x563737bb6490 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.218+0000 7f02887b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563737b17690 con 0x563737bc1480 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bb6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42394/0 (socket says 172.21.15.37:42394) 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bb6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:41318/0 (socket says 172.21.15.37:41318) 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027effd640 1 -- 172.21.15.37:0/2792019298 learned_addr learned my addr 172.21.15.37:0/2792019298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 -- 172.21.15.37:0/2792019298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bc1480 msgr2=0x563737bc3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bc1480 0x563737bc3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 -- 172.21.15.37:0/2792019298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 msgr2=0x563737bc0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 -- 172.21.15.37:0/2792019298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563737ac25b0 con 0x563737bb6490 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027effd640 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f027f7fe640 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bb6870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f02700ce450 tx=0x7f0270009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3823528c4dac24c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f0265ffb640 1 -- 172.21.15.37:0/2792019298 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700a0f80 con 0x563737bb6490 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f0265ffb640 1 -- 172.21.15.37:0/2792019298 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02700a5840 con 0x563737bb6490 2024-04-23T18:21:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.219+0000 7f0265ffb640 1 -- 172.21.15.37:0/2792019298 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700bd070 con 0x563737bb6490 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 -- 172.21.15.37:0/2792019298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 msgr2=0x563737bb6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bb6870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f02700ce450 tx=0x7f0270009510 comp rx=0 tx=0).stop 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 -- 172.21.15.37:0/2792019298 shutdown_connections 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bc1480 0x563737bc3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 --2- 172.21.15.37:0/2792019298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bb6870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.220+0000 7f02887b8740 1 -- 172.21.15.37:0/2792019298 >> 172.21.15.37:0/2792019298 conn(0x563737bb27b0 msgr2=0x563737bb2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.221+0000 7f02887b8740 1 -- 172.21.15.37:0/2792019298 shutdown_connections 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.221+0000 7f02887b8740 1 -- 172.21.15.37:0/2792019298 wait complete. 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.221+0000 7f02887b8740 1 -- start start 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.221+0000 7f02887b8740 1 Processor -- start 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.221+0000 7f02887b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bbe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.221+0000 7f02887b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 0x563737bbe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f02887b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bbece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f02887b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563737bc4310 con 0x563737bc1480 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f02887b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563737ab5b20 con 0x563737bb6db0 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f02887b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563737bc4490 con 0x563737bb6490 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 0x563737bbe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 0x563737bbe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:41334/0 (socket says 172.21.15.37:41334) 2024-04-23T18:21:25.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 -- 172.21.15.37:0/3568185776 learned_addr learned my addr 172.21.15.37:0/3568185776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027ffff640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bbece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027f7fe640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bbe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f02887b8740 10 client.0 _ll_register_callbacks cb 0x563737ab5530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 -- 172.21.15.37:0/3568185776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 msgr2=0x563737bbe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bbe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 -- 172.21.15.37:0/3568185776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 msgr2=0x563737bbece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bbece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 -- 172.21.15.37:0/3568185776 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373782d920 con 0x563737bb6db0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027ffff640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bbece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02700a5840 con 0x563737bb6db0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f027effd640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 0x563737bbe7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0274030af0 tx=0x7f0274005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0cc5d9d0f391a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.222+0000 7f025effd640 10 client.25217 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f025effd640 1 -- 172.21.15.37:0/3568185776 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274005610 con 0x563737bb6db0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f025effd640 1 -- 172.21.15.37:0/3568185776 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f02740046c0 con 0x563737bb6db0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f025effd640 1 -- 172.21.15.37:0/3568185776 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0274004900 con 0x563737bb6db0 2024-04-23T18:21:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f02887b8740 10 client.25217 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5637378f9d30 con 0x563737bb6db0 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f025effd640 1 -- 172.21.15.37:0/3568185776 <== mon.2 v2:172.21.15.172:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0274004aa0 con 0x563737bb6db0 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f02887b8740 10 client.25217 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f02887b8740 10 client.25217 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.223+0000 7f02887b8740 20 client.25217 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.224+0000 7f02887b8740 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737bd39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.224+0000 7f02887b8740 4 client.25217 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.224+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563737ad9500 con 0x563737bd1560 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.224+0000 7f027f7fe640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737bd39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.224+0000 7f025effd640 10 client.25217 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.224+0000 7f027f7fe640 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737bd39d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5466872e5cd13c1d server_cookie=387e96179b66268a in_seq=0 out_seq=0 2024-04-23T18:21:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.225+0000 7f025effd640 1 -- 172.21.15.37:0/3568185776 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563737ad9500 con 0x563737bd1560 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.225+0000 7f025effd640 10 client.25217 handle_command_reply: tid=0 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.225+0000 7f02887b8740 1 client.25217 shutdown 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.225+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 msgr2=0x563737bbe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.225+0000 7f02887b8740 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 0x563737bbe7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0274030af0 tx=0x7f0274005f90 comp rx=0 tx=0).stop 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 shutdown_connections 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737bd39d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bbece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f025effd640 0 client.25217 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6db0 0x563737bbe7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 --2- 172.21.15.37:0/3568185776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bbe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 >> 172.21.15.37:0/3568185776 conn(0x563737bb27b0 msgr2=0x563737bb19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 shutdown_connections 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 1 -- 172.21.15.37:0/3568185776 wait complete. 2024-04-23T18:21:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.226+0000 7f02887b8740 20 client.25217 trim_cache size 0 max 16384 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 Processor -- start 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 -- start start 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bcd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6db0 0x563737bb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bd92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f02801b26f0 con 0x563737bc1480 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02801affc0 con 0x563737bb6490 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.261+0000 7f02887b8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f02801ade10 con 0x563737bb6db0 2024-04-23T18:21:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bcd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bd92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bcd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:41344/0 (socket says 172.21.15.37:41344) 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bd92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42420/0 (socket says 172.21.15.37:42420) 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 -- 172.21.15.37:0/1367898559 learned_addr learned my addr 172.21.15.37:0/1367898559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027effd640 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6db0 0x563737bb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 -- 172.21.15.37:0/1367898559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6db0 msgr2=0x563737bb2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6db0 0x563737bb2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 -- 172.21.15.37:0/1367898559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 msgr2=0x563737bd92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bd92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 -- 172.21.15.37:0/1367898559 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563737bbd670 con 0x563737bb6490 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027effd640 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6db0 0x563737bb2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bcd1d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02700d1190 tx=0x7f027000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad6dd8515786acf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.262+0000 7f02657fa640 1 -- 172.21.15.37:0/1367898559 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270003aa0 con 0x563737bb6490 2024-04-23T18:21:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.263+0000 7f02657fa640 1 -- 172.21.15.37:0/1367898559 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02700037a0 con 0x563737bb6490 2024-04-23T18:21:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.263+0000 7f02657fa640 1 -- 172.21.15.37:0/1367898559 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700ce730 con 0x563737bb6490 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.263+0000 7f02887b8740 1 -- 172.21.15.37:0/1367898559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 msgr2=0x563737bcd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bcd1d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02700d1190 tx=0x7f027000c850 comp rx=0 tx=0).stop 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 -- 172.21.15.37:0/1367898559 shutdown_connections 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bc1480 0x563737bd92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6db0 0x563737bb2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 --2- 172.21.15.37:0/1367898559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bb6490 0x563737bcd1d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 -- 172.21.15.37:0/1367898559 >> 172.21.15.37:0/1367898559 conn(0x563737bb27b0 msgr2=0x563737bbeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 -- 172.21.15.37:0/1367898559 shutdown_connections 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 -- 172.21.15.37:0/1367898559 wait complete. 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 -- start start 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.264+0000 7f02887b8740 1 Processor -- start 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f02887b8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bc2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f02887b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f027effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bc2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f027f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bc2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:33220/0 (socket says 172.21.15.37:33220) 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f027effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42430/0 (socket says 172.21.15.37:42430) 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f02887b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bc1480 0x563737bc7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f027f7fe640 1 -- 172.21.15.37:0/1390949529 learned_addr learned my addr 172.21.15.37:0/1390949529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f02801ae480 con 0x563737bb6db0 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02801af8f0 con 0x563737bc1480 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f02801aeb50 con 0x563737bb6490 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.265+0000 7f027ffff640 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bc1480 0x563737bc7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f02887b8740 10 client.0 _ll_register_callbacks cb 0x7f02740096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f027f7fe640 1 -- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bc1480 msgr2=0x563737bc7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bc1480 0x563737bc7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f027f7fe640 1 -- 172.21.15.37:0/1390949529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 msgr2=0x563737bc2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f027f7fe640 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563737bc81b0 con 0x563737bb6490 2024-04-23T18:21:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02700037a0 con 0x563737bb6490 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f027f7fe640 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bc2630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f027009ae80 tx=0x7f0270003950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360f09e9526c14bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f025effd640 10 client.25201 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f025effd640 1 -- 172.21.15.37:0/1390949529 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270003aa0 con 0x563737bb6490 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f025effd640 1 -- 172.21.15.37:0/1390949529 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f027001f340 con 0x563737bb6490 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.266+0000 7f025effd640 1 -- 172.21.15.37:0/1390949529 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02700ce730 con 0x563737bb6490 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 10 client.25201 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:25.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({fsmap=25}) -- 0x563737c380a0 con 0x563737bb6490 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f025effd640 1 -- 172.21.15.37:0/1390949529 <== mon.1 v2:172.21.15.173:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0274030630 con 0x563737bb6490 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 10 client.25201 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 10 client.25201 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 20 client.25201 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737c3a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 4 client.25201 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5ec38c87", "release": true, "await": true}] 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5ec38c87", "release": true, "await": true}) -- 0x563737b290f0 con 0x563737bd1560 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.267+0000 7f027effd640 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737c3a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.268+0000 7f027effd640 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737c3a280 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867818ded373614a server_cookie=965932971aae4a69 in_seq=0 out_seq=0 2024-04-23T18:21:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.268+0000 7f025effd640 10 client.25201 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.269+0000 7f025effd640 1 -- 172.21.15.37:0/1390949529 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x563737b290f0 con 0x563737bd1560 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.269+0000 7f025effd640 10 client.25201 handle_command_reply: tid=0 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.269+0000 7f02887b8740 1 client.25201 shutdown 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 msgr2=0x563737bc2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bc2630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f027009ae80 tx=0x7f0270003950 comp rx=0 tx=0).stop 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 shutdown_connections 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563737bd1560 0x563737c3a280 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563737bc1480 0x563737bc7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563737bb6db0 0x563737bc2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f025effd640 0 client.25201 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 --2- 172.21.15.37:0/1390949529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563737bb6490 0x563737bc2630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 >> 172.21.15.37:0/1390949529 conn(0x563737bb27b0 msgr2=0x563737bbfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 shutdown_connections 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 1 -- 172.21.15.37:0/1390949529 wait complete. 2024-04-23T18:21:25.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f02887b8740 20 client.25201 trim_cache size 0 max 16384 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 4, 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "5ec38c87": { 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4, 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:21:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.3, 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.267 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:21:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02680a3ec0 msgr2=0x7f02680a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f0285aa0640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02680a3ec0 0x7f02680a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f028010e360 tx=0x7f0274005c90 comp rx=0 tx=0).stop 2024-04-23T18:21:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 msgr2=0x7f0280198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.270+0000 7f0285aa0640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 0x7f0280198750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f027000e9c0 tx=0x7f027000ee80 comp rx=0 tx=0).stop 2024-04-23T18:21:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 shutdown_connections 2024-04-23T18:21:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02680a3ec0 0x7f02680a6380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028010c510 0x7f028019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0280103af0 0x7f0280198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 --2- 172.21.15.37:0/1347822780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02801031d0 0x7f0280198750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 >> 172.21.15.37:0/1347822780 conn(0x7f0280077f50 msgr2=0x7f02800fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 shutdown_connections 2024-04-23T18:21:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.271+0000 7f0285aa0640 1 -- 172.21.15.37:0/1347822780 wait complete. 2024-04-23T18:21:25.278 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5ec38c87', seconds elapsed: 30.4 2024-04-23T18:21:25.278 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.3 2024-04-23T18:21:25.278 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:21:25.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 -- 172.21.15.37:0/2812060555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc103b00 msgr2=0x7f20fc10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 --2- 172.21.15.37:0/2812060555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc103b00 0x7f20fc10bfe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009900 tx=0x7f20e402a510 comp rx=0 tx=0).stop 2024-04-23T18:21:25.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 -- 172.21.15.37:0/2812060555 shutdown_connections 2024-04-23T18:21:25.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 --2- 172.21.15.37:0/2812060555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20fc10c520 0x7f20fc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 --2- 172.21.15.37:0/2812060555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc103b00 0x7f20fc10bfe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 --2- 172.21.15.37:0/2812060555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc1031e0 0x7f20fc1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.389+0000 7f2100823640 1 -- 172.21.15.37:0/2812060555 >> 172.21.15.37:0/2812060555 conn(0x7f20fc077f50 msgr2=0x7f20fc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.390+0000 7f2100823640 1 -- 172.21.15.37:0/2812060555 shutdown_connections 2024-04-23T18:21:25.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.390+0000 7f2100823640 1 -- 172.21.15.37:0/2812060555 wait complete. 2024-04-23T18:21:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.390+0000 7f2100823640 1 Processor -- start 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 -- start start 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20fc1031e0 0x7f20fc100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc103b00 0x7f20fc101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 0x7f20fc1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc112190 con 0x7f20fc10c520 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc112010 con 0x7f20fc103b00 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f2100823640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc112310 con 0x7f20fc1031e0 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 0x7f20fc1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 0x7f20fc1018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42452/0 (socket says 172.21.15.37:42452) 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 -- 172.21.15.37:0/293487441 learned_addr learned my addr 172.21.15.37:0/293487441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20f1d9b640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc103b00 0x7f20fc101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fa59c640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20fc1031e0 0x7f20fc100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 -- 172.21.15.37:0/293487441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20fc1031e0 msgr2=0x7f20fc100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20fc1031e0 0x7f20fc100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 -- 172.21.15.37:0/293487441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc103b00 msgr2=0x7f20fc101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc103b00 0x7f20fc101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.391+0000 7f20fad9d640 1 -- 172.21.15.37:0/293487441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc1a6c30 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.392+0000 7f20fad9d640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 0x7f20fc1018b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f20f400cad0 tx=0x7f20f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.392+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400ae20 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.392+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc1a6f20 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.392+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f4005020 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.392+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400f6b0 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.392+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc1a73f0 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.393+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc068d70 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.394+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f20f40051d0 con 0x7f20fc10c520 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.395+0000 7f20f3fff640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f20e80a3e40 0x7f20e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.395+0000 7f20fa59c640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f20e80a3e40 0x7f20e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.395+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7f20f40d0ae0 con 0x7f20fc10c520 2024-04-23T18:21:25.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.396+0000 7f20fa59c640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f20e80a3e40 0x7f20e80a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00bbe0 tx=0x7f20ec00b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.400+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20f4014030 con 0x7f20fc10c520 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 -- 172.21.15.37:0/2030907778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841062f0 msgr2=0x7fc284106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2030907778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841062f0 0x7fc284106780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009ca0 tx=0x7fc27c02a9e0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 -- 172.21.15.37:0/2030907778 shutdown_connections 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2030907778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841062f0 0x7fc284106780 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2030907778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2841018f0 0x7fc284105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2030907778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 0x7fc2841013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 -- 172.21.15.37:0/2030907778 >> 172.21.15.37:0/2030907778 conn(0x7fc284077fd0 msgr2=0x7fc2840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 -- 172.21.15.37:0/2030907778 shutdown_connections 2024-04-23T18:21:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 -- 172.21.15.37:0/2030907778 wait complete. 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 Processor -- start 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 -- start start 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.497+0000 7fc28ace1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 0x7fc2841a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc28ace1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 0x7fc2841a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc28ace1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2841062f0 0x7fc28419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc28ace1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841187b0 con 0x7fc2841018f0 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc28ace1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc284118630 con 0x7fc2841062f0 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc28ace1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc284118930 con 0x7fc284100fd0 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 0x7fc2841a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 0x7fc2841a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42472/0 (socket says 172.21.15.37:42472) 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc283fff640 1 -- 172.21.15.37:0/2932634824 learned_addr learned my addr 172.21.15.37:0/2932634824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 0x7fc2841a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc28927e640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2841062f0 0x7fc28419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 -- 172.21.15.37:0/2932634824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2841062f0 msgr2=0x7fc28419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2841062f0 0x7fc28419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 -- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 msgr2=0x7fc2841a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 0x7fc2841a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 -- 172.21.15.37:0/2932634824 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28419f230 con 0x7fc284100fd0 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc283fff640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 0x7fc2841a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.498+0000 7fc288a7d640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 0x7fc2841a4900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d720 tx=0x7fc27800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.499+0000 7fc281ffb640 1 -- 172.21.15.37:0/2932634824 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278017070 con 0x7fc284100fd0 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.499+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28419f520 con 0x7fc284100fd0 2024-04-23T18:21:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.499+0000 7fc281ffb640 1 -- 172.21.15.37:0/2932634824 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27800de30 con 0x7fc284100fd0 2024-04-23T18:21:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.499+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2841ab7c0 con 0x7fc284100fd0 2024-04-23T18:21:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.499+0000 7fc281ffb640 1 -- 172.21.15.37:0/2932634824 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780047a0 con 0x7fc284100fd0 2024-04-23T18:21:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.502+0000 7fc281ffb640 1 -- 172.21.15.37:0/2932634824 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fc2780099d0 con 0x7fc284100fd0 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 Processor -- start 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 -- start start 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc281ffb640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc26c0a3f60 0x7fc26c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135b620 0x55741135ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 0x5574113662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411368bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5574112c0970 con 0x557411366800 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55741125d4a0 con 0x55741135bf40 2024-04-23T18:21:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5574112bc930 con 0x55741135b620 2024-04-23T18:21:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc283fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 0x5574113662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc283fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 0x5574113662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:41384/0 (socket says 172.21.15.37:41384) 2024-04-23T18:21:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc283fff640 1 -- 172.21.15.37:0/3216565026 learned_addr learned my addr 172.21.15.37:0/3216565026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc288a7d640 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135b620 0x55741135ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.503+0000 7fc28927e640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc26c0a3f60 0x7fc26c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc283fff640 1 -- 172.21.15.37:0/3216565026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135b620 msgr2=0x55741135ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc283fff640 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135b620 0x55741135ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc283fff640 1 -- 172.21.15.37:0/3216565026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 msgr2=0x557411368bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc283fff640 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411368bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc283fff640 1 -- 172.21.15.37:0/3216565026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557410fd3210 con 0x55741135bf40 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc281ffb640 1 -- 172.21.15.37:0/2932634824 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc2780d06d0 con 0x7fc284100fd0 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc283fff640 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 0x5574113662c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2740104d0 tx=0x7fc2740108d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a84d9fc65f4fe68 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc26affd640 1 -- 172.21.15.37:0/3216565026 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274005220 con 0x55741135bf40 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc26affd640 1 -- 172.21.15.37:0/3216565026 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc274004c50 con 0x55741135bf40 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc26affd640 1 -- 172.21.15.37:0/3216565026 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274011a70 con 0x55741135bf40 2024-04-23T18:21:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.504+0000 7fc28927e640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc26c0a3f60 0x7fc26c0a6420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc27c006010 tx=0x7fc27c03b040 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3216565026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 msgr2=0x5574113662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 0x5574113662c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2740104d0 tx=0x7fc2740108d0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3216565026 shutdown_connections 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411368bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135bf40 0x5574113662c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3216565026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135b620 0x55741135ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3216565026 >> 172.21.15.37:0/3216565026 conn(0x557411357b30 msgr2=0x557411357f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3216565026 shutdown_connections 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3216565026 wait complete. 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 -- start start 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.505+0000 7fc28d9f6740 1 Processor -- start 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x557411366220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 0x557411363510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557411366800 0x557411363b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557411369430 con 0x55741135bf40 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55741125b0b0 con 0x55741135b620 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5574113695b0 con 0x557411366800 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28d9f6740 10 client.0 _ll_register_callbacks cb 0x557411248d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 0x557411363510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc288a7d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x557411366220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 0x557411363510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42488/0 (socket says 172.21.15.37:42488) 2024-04-23T18:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc283fff640 1 -- 172.21.15.37:0/559034132 learned_addr learned my addr 172.21.15.37:0/559034132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc28927e640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557411366800 0x557411363b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc288a7d640 1 -- 172.21.15.37:0/559034132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557411366800 msgr2=0x557411363b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc288a7d640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557411366800 0x557411363b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc288a7d640 1 -- 172.21.15.37:0/559034132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 msgr2=0x557411363510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc288a7d640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 0x557411363510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.506+0000 7fc288a7d640 1 -- 172.21.15.37:0/559034132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55741109f620 con 0x55741135b620 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28927e640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557411366800 0x557411363b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc283fff640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 0x557411363510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc288a7d640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x557411366220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc2780cdc50 tx=0x7fc278066f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2003ef84c4b4adf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc263fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc263fff640 1 -- 172.21.15.37:0/559034132 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278005020 con 0x55741135b620 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc274004c50 con 0x55741135b620 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc263fff640 1 -- 172.21.15.37:0/559034132 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc278098a40 con 0x55741135b620 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc263fff640 1 -- 172.21.15.37:0/559034132 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc27809cad0 con 0x55741135b620 2024-04-23T18:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28d9f6740 10 client.25241 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557411375470 con 0x55741135b620 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc263fff640 1 -- 172.21.15.37:0/559034132 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc274004c50 con 0x55741135b620 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc263fff640 1 -- 172.21.15.37:0/559034132 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2780a4c40 con 0x55741135b620 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28d9f6740 10 client.25241 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28d9f6740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.507+0000 7fc28d9f6740 20 client.25241 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.508+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557411376930 0x557411378da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.508+0000 7fc28d9f6740 4 client.25241 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.508+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574112ce390 con 0x557411376930 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.508+0000 7fc283fff640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557411376930 0x557411378da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.508+0000 7fc283fff640 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557411376930 0x557411378da0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43b148654978871 server_cookie=2da7d77715c6df87 in_seq=0 out_seq=0 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.508+0000 7fc263fff640 10 client.25241 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc263fff640 1 -- 172.21.15.37:0/559034132 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574112ce390 con 0x557411376930 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc263fff640 10 client.25241 handle_command_reply: tid=0 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 client.25241 shutdown 2024-04-23T18:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 msgr2=0x557411366220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x557411366220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc2780cdc50 tx=0x7fc278066f20 comp rx=0 tx=0).stop 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 shutdown_connections 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557411376930 0x557411378da0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc263fff640 0 client.25241 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557411366800 0x557411363b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135bf40 0x557411363510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/559034132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x557411366220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 >> 172.21.15.37:0/559034132 conn(0x557411357b30 msgr2=0x55741135d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 shutdown_connections 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 1 -- 172.21.15.37:0/559034132 wait complete. 2024-04-23T18:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.509+0000 7fc28d9f6740 20 client.25241 trim_cache size 0 max 16384 2024-04-23T18:21:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 Processor -- start 2024-04-23T18:21:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 -- start start 2024-04-23T18:21:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x5574113722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x557411356b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411365d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841b6960 con 0x557411366800 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841b4230 con 0x55741135b620 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.524+0000 7fc28d9f6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841b2080 con 0x55741135bf40 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc28927e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411365d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x5574113722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc28927e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411365d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42502/0 (socket says 172.21.15.37:42502) 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc28927e640 1 -- 172.21.15.37:0/3494885403 learned_addr learned my addr 172.21.15.37:0/3494885403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc283fff640 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x557411356b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 -- 172.21.15.37:0/3494885403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 msgr2=0x557411356b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x557411356b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 -- 172.21.15.37:0/3494885403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 msgr2=0x557411365d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411365d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 -- 172.21.15.37:0/3494885403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557411362520 con 0x55741135b620 2024-04-23T18:21:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc283fff640 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x557411356b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc28927e640 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411365d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc288a7d640 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x5574113722f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc278098da0 tx=0x7fc27809a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2e629217dce72a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.525+0000 7fc26a7fc640 1 -- 172.21.15.37:0/3494885403 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278014070 con 0x55741135b620 2024-04-23T18:21:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc26a7fc640 1 -- 172.21.15.37:0/3494885403 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc278005020 con 0x55741135b620 2024-04-23T18:21:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc26a7fc640 1 -- 172.21.15.37:0/3494885403 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780a5040 con 0x55741135b620 2024-04-23T18:21:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3494885403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 msgr2=0x5574113722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x5574113722f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc278098da0 tx=0x7fc27809a8e0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3494885403 shutdown_connections 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557411366800 0x557411365d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x557411356b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/3494885403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55741135b620 0x5574113722f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3494885403 >> 172.21.15.37:0/3494885403 conn(0x557411357b30 msgr2=0x557411356420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.526+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3494885403 shutdown_connections 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.527+0000 7fc28d9f6740 1 -- 172.21.15.37:0/3494885403 wait complete. 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.527+0000 7fc28d9f6740 1 -- start start 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.527+0000 7fc28d9f6740 1 Processor -- start 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.527+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 0x55741136eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.527+0000 7fc28d9f6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x55741135fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 0x55741136eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 0x55741136eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42518/0 (socket says 172.21.15.37:42518) 2024-04-23T18:21:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 -- 172.21.15.37:0/1353180973 learned_addr learned my addr 172.21.15.37:0/1353180973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557411366800 0x557411360420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841b26f0 con 0x55741135b620 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841b3b60 con 0x557411366800 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841b2dc0 con 0x55741135bf40 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc283fff640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x55741135fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc28927e640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557411366800 0x557411360420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 -- 172.21.15.37:0/1353180973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 msgr2=0x55741135fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x55741135fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 -- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557411366800 msgr2=0x557411360420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557411366800 0x557411360420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.528+0000 7fc288a7d640 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557411360c20 con 0x55741135b620 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc288a7d640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 0x55741136eb80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc2780cdff0 tx=0x7fc27809ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcebbb253d24cc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc263fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc263fff640 1 -- 172.21.15.37:0/1353180973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780b5060 con 0x55741135b620 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc263fff640 1 -- 172.21.15.37:0/1353180973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fc27809b290 con 0x55741135b620 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc263fff640 10 client.0 inactive, discarding osd_map(91..91 src has 1..91) 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc28d9f6740 10 client.0 _ll_register_callbacks cb 0x7fc278004220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5574112c8200 con 0x55741135b620 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc263fff640 1 -- 172.21.15.37:0/1353180973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc278066050 con 0x55741135b620 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc28d9f6740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.529+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5574113dcb50 con 0x55741135b620 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc263fff640 1 -- 172.21.15.37:0/1353180973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2780a1910 con 0x55741135b620 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc28d9f6740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc28d9f6740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc28d9f6740 20 client.15654 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5574113dcf70 0x5574113df390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc28d9f6740 4 client.15654 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:21:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574112445f0 con 0x5574113dcf70 2024-04-23T18:21:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc283fff640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5574113dcf70 0x5574113df390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc283fff640 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5574113dcf70 0x5574113df390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795f17cebeb0e91 server_cookie=a151a9ea6b908e4d in_seq=0 out_seq=0 2024-04-23T18:21:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.530+0000 7fc263fff640 10 client.15654 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc263fff640 1 -- 172.21.15.37:0/1353180973 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574112445f0 con 0x5574113dcf70 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc263fff640 10 client.15654 handle_command_reply: tid=0 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 client.15654 shutdown 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 msgr2=0x55741136eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 0x55741136eb80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc2780cdff0 tx=0x7fc27809ba50 comp rx=0 tx=0).stop 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 shutdown_connections 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5574113dcf70 0x5574113df390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557411366800 0x557411360420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55741135bf40 0x55741135fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc263fff640 0 client.15654 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 --2- 172.21.15.37:0/1353180973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55741135b620 0x55741136eb80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.533+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 >> 172.21.15.37:0/1353180973 conn(0x557411357b30 msgr2=0x557411356cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 shutdown_connections 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28d9f6740 1 -- 172.21.15.37:0/1353180973 wait complete. 2024-04-23T18:21:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28d9f6740 20 client.15654 trim_cache size 0 max 16384 2024-04-23T18:21:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:21:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:21:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:21:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc26c0a3f60 msgr2=0x7fc26c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc26c0a3f60 0x7fc26c0a6420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc27c006010 tx=0x7fc27c03b040 comp rx=0 tx=0).stop 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 msgr2=0x7fc2841a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 0x7fc2841a4900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d720 tx=0x7fc27800dbe0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 shutdown_connections 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc26c0a3f60 0x7fc26c0a6420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2841062f0 0x7fc28419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2841018f0 0x7fc2841a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 --2- 172.21.15.37:0/2932634824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc284100fd0 0x7fc2841a4900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.534+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 >> 172.21.15.37:0/2932634824 conn(0x7fc284077fd0 msgr2=0x7fc284103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.535+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 shutdown_connections 2024-04-23T18:21:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:25.535+0000 7fc28ace1640 1 -- 172.21.15.37:0/2932634824 wait complete. 2024-04-23T18:21:25.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.693+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20fc1076a0 con 0x7f20fc10c520 2024-04-23T18:21:25.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.694+0000 7f20f3fff640 1 -- 172.21.15.37:0/293487441 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11878 (secure 0 0 0) 0x7f20f409c890 con 0x7f20fc10c520 2024-04-23T18:21:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:21:25.687 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:58.538288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:21:25.687 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-23T18:21:25.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.696+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f20e80a3e40 msgr2=0x7f20e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.696+0000 7f2100823640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f20e80a3e40 0x7f20e80a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00bbe0 tx=0x7f20ec00b390 comp rx=0 tx=0).stop 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.696+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 msgr2=0x7f20fc1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.696+0000 7f2100823640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 0x7f20fc1018b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f20f400cad0 tx=0x7f20f400cf90 comp rx=0 tx=0).stop 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 shutdown_connections 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f20e80a3e40 0x7f20e80a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc10c520 0x7f20fc1018b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc103b00 0x7f20fc101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 --2- 172.21.15.37:0/293487441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20fc1031e0 0x7f20fc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 >> 172.21.15.37:0/293487441 conn(0x7f20fc077f50 msgr2=0x7f20fc0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 shutdown_connections 2024-04-23T18:21:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.697+0000 7f2100823640 1 -- 172.21.15.37:0/293487441 wait complete. 2024-04-23T18:21:25.701 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 89.3 --await-for 42.0' 2024-04-23T18:21:25.701 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 89.3 --await-for 42.0 2024-04-23T18:21:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.824+0000 7fab942ae640 1 -- 172.21.15.37:0/56089485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 msgr2=0x7fab8c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.824+0000 7fab942ae640 1 --2- 172.21.15.37:0/56089485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c100d70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009900 tx=0x7fab7c02a400 comp rx=0 tx=0).stop 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 -- 172.21.15.37:0/56089485 shutdown_connections 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 --2- 172.21.15.37:0/56089485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c1012b0 0x7fab8c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 --2- 172.21.15.37:0/56089485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c100d70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 --2- 172.21.15.37:0/56089485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c108070 0x7fab8c108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 -- 172.21.15.37:0/56089485 >> 172.21.15.37:0/56089485 conn(0x7fab8c0760f0 msgr2=0x7fab8c076500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 -- 172.21.15.37:0/56089485 shutdown_connections 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 -- 172.21.15.37:0/56089485 wait complete. 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 Processor -- start 2024-04-23T18:21:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.825+0000 7fab942ae640 1 -- start start 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab942ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab942ae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c1012b0 0x7fab8c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab942ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c108070 0x7fab8c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab942ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c112230 con 0x7fab8c1008f0 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab942ae640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1120b0 con 0x7fab8c108070 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab942ae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1123b0 con 0x7fab8c1012b0 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42542/0 (socket says 172.21.15.37:42542) 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 -- 172.21.15.37:0/2681022873 learned_addr learned my addr 172.21.15.37:0/2681022873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9284b640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c108070 0x7fab8c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab91849640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c1012b0 0x7fab8c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 -- 172.21.15.37:0/2681022873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c1012b0 msgr2=0x7fab8c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c1012b0 0x7fab8c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 -- 172.21.15.37:0/2681022873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c108070 msgr2=0x7fab8c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c108070 0x7fab8c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 -- 172.21.15.37:0/2681022873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c198fc0 con 0x7fab8c1008f0 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9284b640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c108070 0x7fab8c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.826+0000 7fab9204a640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c19e640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab8800e9f0 tx=0x7fab8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.827+0000 7fab837fe640 1 -- 172.21.15.37:0/2681022873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8800cd80 con 0x7fab8c1008f0 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.827+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1992b0 con 0x7fab8c1008f0 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.827+0000 7fab837fe640 1 -- 172.21.15.37:0/2681022873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab880108e0 con 0x7fab8c1008f0 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.827+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a55c0 con 0x7fab8c1008f0 2024-04-23T18:21:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.828+0000 7fab837fe640 1 -- 172.21.15.37:0/2681022873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88018980 con 0x7fab8c1008f0 2024-04-23T18:21:25.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 Processor -- start 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 -- start start 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980df8f80 0x55f980df9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e07f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980e084a0 0x55f980e0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f980d5e370 con 0x55f980e084a0 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f980cfb2e0 con 0x55f980df98a0 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.830+0000 7fab96fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f980d5a330 con 0x55f980df8f80 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab837fe640 1 -- 172.21.15.37:0/2681022873 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fab88010430 con 0x7fab8c1008f0 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab91849640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980e084a0 0x55f980e0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab91849640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980e084a0 0x55f980e0a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42556/0 (socket says 172.21.15.37:42556) 2024-04-23T18:21:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab91849640 1 -- 172.21.15.37:0/2568864678 learned_addr learned my addr 172.21.15.37:0/2568864678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980df8f80 0x55f980df9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9204a640 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e07f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 -- 172.21.15.37:0/2568864678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 msgr2=0x55f980e07f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e07f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 -- 172.21.15.37:0/2568864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980e084a0 msgr2=0x55f980e0a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980e084a0 0x55f980e0a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 -- 172.21.15.37:0/2568864678 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f980b3cd80 con 0x55f980df8f80 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9204a640 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e07f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.831+0000 7fab9284b640 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980df8f80 0x55f980df9360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab84009f80 tx=0x7fab8400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752c15b2edda9f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab80bfe640 1 -- 172.21.15.37:0/2568864678 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84015070 con 0x55f980df8f80 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab80bfe640 1 -- 172.21.15.37:0/2568864678 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84011bb0 con 0x55f980df8f80 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab80bfe640 1 -- 172.21.15.37:0/2568864678 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8401ac50 con 0x55f980df8f80 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab837fe640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fab780a3ec0 0x7fab780a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab9204a640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fab780a3ec0 0x7fab780a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab96fc6740 1 -- 172.21.15.37:0/2568864678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980df8f80 msgr2=0x55f980df9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.832+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980df8f80 0x55f980df9360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab84009f80 tx=0x7fab8400f7f0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 -- 172.21.15.37:0/2568864678 shutdown_connections 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980e084a0 0x55f980e0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e07f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2568864678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980df8f80 0x55f980df9360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 -- 172.21.15.37:0/2568864678 >> 172.21.15.37:0/2568864678 conn(0x55f980df53c0 msgr2=0x55f980df57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab837fe640 1 -- 172.21.15.37:0/2681022873 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fab880d0dd0 con 0x7fab8c1008f0 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 -- 172.21.15.37:0/2568864678 shutdown_connections 2024-04-23T18:21:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 -- 172.21.15.37:0/2568864678 wait complete. 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 -- start start 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.833+0000 7fab96fc6740 1 Processor -- start 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9204a640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fab780a3ec0 0x7fab780a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab880ce410 tx=0x7fab88000bc0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42568/0 (socket says 172.21.15.37:42568) 2024-04-23T18:21:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 -- 172.21.15.37:0/2923907336 learned_addr learned my addr 172.21.15.37:0/2923907336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f980e0b2e0 con 0x55f980df8f80 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f980cf8fb0 con 0x55f980df98a0 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f980e0b460 con 0x55f980e084a0 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab91849640 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab96fc6740 10 client.0 _ll_register_callbacks cb 0x55f980cf89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 -- 172.21.15.37:0/2923907336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 msgr2=0x55f980e05e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 -- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 msgr2=0x55f980e05920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e05920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.834+0000 7fab9284b640 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f980b9d510 con 0x55f980df8f80 2024-04-23T18:21:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab9284b640 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e053e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fab8401a810 tx=0x7fab84012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad66fde5d3d8bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab84011bb0 con 0x55f980df8f80 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab75ffb640 1 -- 172.21.15.37:0/2923907336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84029060 con 0x55f980df8f80 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab75ffb640 1 -- 172.21.15.37:0/2923907336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fab84004c50 con 0x55f980df8f80 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab75ffb640 1 -- 172.21.15.37:0/2923907336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab84015050 con 0x55f980df8f80 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55f980a30670 con 0x55f980df8f80 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab75ffb640 1 -- 172.21.15.37:0/2923907336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab8401a9f0 con 0x55f980df8f80 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 10 client.15666 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 20 client.15666 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e185e0 0x55f980e1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 4 client.15666 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.835+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab84011bb0 con 0x55f980e185e0 2024-04-23T18:21:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.836+0000 7fab91849640 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e185e0 0x55f980e1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.836+0000 7fab91849640 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e185e0 0x55f980e1a9d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464e0bc842980e86 server_cookie=8f6baec50f91b29a in_seq=0 out_seq=0 2024-04-23T18:21:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.836+0000 7fab75ffb640 10 client.15666 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab75ffb640 1 -- 172.21.15.37:0/2923907336 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fab84011bb0 con 0x55f980e185e0 2024-04-23T18:21:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab75ffb640 10 client.15666 handle_command_reply: tid=0 2024-04-23T18:21:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 client.15666 shutdown 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 msgr2=0x55f980e053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e053e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fab8401a810 tx=0x7fab84012d70 comp rx=0 tx=0).stop 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 shutdown_connections 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e185e0 0x55f980e1a9d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e05920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 --2- 172.21.15.37:0/2923907336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e053e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 >> 172.21.15.37:0/2923907336 conn(0x55f980df53c0 msgr2=0x55f980dfb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab75ffb640 0 client.15666 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 shutdown_connections 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 1 -- 172.21.15.37:0/2923907336 wait complete. 2024-04-23T18:21:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.837+0000 7fab96fc6740 20 client.15666 trim_cache size 0 max 16384 2024-04-23T18:21:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.862+0000 7fab96fc6740 1 Processor -- start 2024-04-23T18:21:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.862+0000 7fab96fc6740 1 -- start start 2024-04-23T18:21:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df8f80 0x55f980e14240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 0x55f980e04cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab96fc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ae3c0 con 0x55f980df98a0 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab96fc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1abc90 con 0x55f980df8f80 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab96fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1a9ae0 con 0x55f980e084a0 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 0x55f980e04cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 0x55f980e04cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42582/0 (socket says 172.21.15.37:42582) 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 -- 172.21.15.37:0/3001031134 learned_addr learned my addr 172.21.15.37:0/3001031134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab9284b640 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab9284b640 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df8f80 0x55f980e14240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 -- 172.21.15.37:0/3001031134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 msgr2=0x55f980e05ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 -- 172.21.15.37:0/3001031134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df8f80 msgr2=0x55f980e14240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df8f80 0x55f980e14240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 -- 172.21.15.37:0/3001031134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f980df6580 con 0x55f980df98a0 2024-04-23T18:21:25.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.863+0000 7fab91849640 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 0x55f980e04cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51241034e8b92826 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.864+0000 7fab77fff640 1 -- 172.21.15.37:0/3001031134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c049070 con 0x55f980df98a0 2024-04-23T18:21:25.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.864+0000 7fab77fff640 1 -- 172.21.15.37:0/3001031134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c044dc0 con 0x55f980df98a0 2024-04-23T18:21:25.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.864+0000 7fab77fff640 1 -- 172.21.15.37:0/3001031134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c035040 con 0x55f980df98a0 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 -- 172.21.15.37:0/3001031134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 msgr2=0x55f980e04cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 0x55f980e04cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000c00 tx=0x7fab7c005fd0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 -- 172.21.15.37:0/3001031134 shutdown_connections 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e05ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df98a0 0x55f980e04cc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 --2- 172.21.15.37:0/3001031134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df8f80 0x55f980e14240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 -- 172.21.15.37:0/3001031134 >> 172.21.15.37:0/3001031134 conn(0x55f980df53c0 msgr2=0x55f980e053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 -- 172.21.15.37:0/3001031134 shutdown_connections 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 -- 172.21.15.37:0/3001031134 wait complete. 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 -- start start 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.865+0000 7fab96fc6740 1 Processor -- start 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e01f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1aa150 con 0x55f980df8f80 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1ab5c0 con 0x55f980df98a0 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1aa820 con 0x55f980e084a0 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e01f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab96fc6740 10 client.0 _ll_register_callbacks cb 0x7fab8401a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e01f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42594/0 (socket says 172.21.15.37:42594) 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 -- 172.21.15.37:0/1409994747 learned_addr learned my addr 172.21.15.37:0/1409994747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab91849640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 -- 172.21.15.37:0/1409994747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 msgr2=0x55f980e02a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e02a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 -- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 msgr2=0x55f980e024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 -- 172.21.15.37:0/1409994747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f980e7d7e0 con 0x55f980df8f80 2024-04-23T18:21:25.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab9284b640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e01f90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab8401a620 tx=0x7fab84021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590170e5773174bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.866+0000 7fab75ffb640 1 -- 172.21.15.37:0/1409994747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84003970 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab75ffb640 1 -- 172.21.15.37:0/1409994747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8506+0+0 (secure 0 0 0) 0x7fab84003b10 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7c044dc0 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab75ffb640 1 -- 172.21.15.37:0/1409994747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab84029020 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 10 client.15672 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f980e7ea00 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab75ffb640 1 -- 172.21.15.37:0/1409994747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c044dc0 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab75ffb640 1 -- 172.21.15.37:0/1409994747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab84005490 con 0x55f980df8f80 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 10 client.15672 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 10 client.15672 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 20 client.15672 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e7f190 0x55f980e815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 4 client.15672 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.3, "await_for": 42.0}] 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.867+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.3, "await_for": 42.0}) -- 0x55f980d6bd90 con 0x55f980e7f190 2024-04-23T18:21:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.868+0000 7fab91849640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e7f190 0x55f980e815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:25.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.868+0000 7fab75ffb640 10 client.15672 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.868+0000 7fab91849640 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e7f190 0x55f980e815b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9109bce06879dab6 server_cookie=b967eea5c72cb95b in_seq=0 out_seq=0 2024-04-23T18:21:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.874+0000 7fab75ffb640 1 -- 172.21.15.37:0/1409994747 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f980d6bd90 con 0x55f980e7f190 2024-04-23T18:21:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.874+0000 7fab75ffb640 10 client.15672 handle_command_reply: tid=0 2024-04-23T18:21:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 client.15672 shutdown 2024-04-23T18:21:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 msgr2=0x55f980e01f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e01f90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab8401a620 tx=0x7fab84021000 comp rx=0 tx=0).stop 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 shutdown_connections 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f980e7f190 0x55f980e815b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f980e084a0 0x55f980e02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab75ffb640 0 client.15672 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f980df98a0 0x55f980e024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 --2- 172.21.15.37:0/1409994747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f980df8f80 0x55f980e01f90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 >> 172.21.15.37:0/1409994747 conn(0x55f980df53c0 msgr2=0x55f980dfb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 shutdown_connections 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 1 -- 172.21.15.37:0/1409994747 wait complete. 2024-04-23T18:21:25.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.875+0000 7fab96fc6740 20 client.15672 trim_cache size 0 max 16384 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 6, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "1d7c0ffc": { 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 6, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.3, 2024-04-23T18:21:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:25.871 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fab780a3ec0 msgr2=0x7fab780a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fab780a3ec0 0x7fab780a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab880ce410 tx=0x7fab88000bc0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 msgr2=0x7fab8c19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c19e640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab8800e9f0 tx=0x7fab8800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 shutdown_connections 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fab780a3ec0 0x7fab780a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab8c108070 0x7fab8c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab8c1012b0 0x7fab8c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 --2- 172.21.15.37:0/2681022873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab8c1008f0 0x7fab8c19e640 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 >> 172.21.15.37:0/2681022873 conn(0x7fab8c0760f0 msgr2=0x7fab8c0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 shutdown_connections 2024-04-23T18:21:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:25.876+0000 7fab942ae640 1 -- 172.21.15.37:0/2681022873 wait complete. 2024-04-23T18:21:25.884 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1d7c0ffc', quiesce took 0.6 seconds. Will release after: 29.3 2024-04-23T18:21:26.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:25 smithi037 ceph-mon[29404]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T18:21:26.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:25 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/4052932241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:26.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:25 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/293487441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:25 smithi172 ceph-mon[32456]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T18:21:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:25 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/4052932241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:25 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/293487441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:26.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:25 smithi173 ceph-mon[32906]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T18:21:26.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:25 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/4052932241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:26.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:25 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/293487441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:28.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:27 smithi172 ceph-mon[32456]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:21:28.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:27 smithi173 ceph-mon[32906]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:21:28.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:27 smithi037 ceph-mon[29404]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:21:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:29 smithi172 ceph-mon[32456]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-23T18:21:30.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:29 smithi173 ceph-mon[32906]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-23T18:21:30.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:29 smithi037 ceph-mon[29404]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-23T18:21:31.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:30 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-23T18:21:31.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:30 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-23T18:21:31.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:30 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-23T18:21:32.271 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:31 smithi037 ceph-mon[29404]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:21:32.271 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:31 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-23T18:21:32.271 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:31 smithi037 ceph-mon[29404]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:21:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:31 smithi172 ceph-mon[32456]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:21:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:31 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-23T18:21:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:31 smithi172 ceph-mon[32456]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:21:32.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:31 smithi173 ceph-mon[32906]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:21:32.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:31 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-23T18:21:32.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:31 smithi173 ceph-mon[32906]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:21:33.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:32 smithi172 ceph-mon[32456]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:21:33.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:32 smithi173 ceph-mon[32906]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:21:33.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:32 smithi037 ceph-mon[29404]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:21:34.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:33 smithi172 ceph-mon[32456]: pgmap v36: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-23T18:21:34.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:33 smithi172 ceph-mon[32456]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-23T18:21:34.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:33 smithi173 ceph-mon[32906]: pgmap v36: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-23T18:21:34.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:33 smithi173 ceph-mon[32906]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-23T18:21:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:33 smithi037 ceph-mon[29404]: pgmap v36: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-23T18:21:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:33 smithi037 ceph-mon[29404]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to '57e0caf76fdb7ab8b1358588f292d08519163844'. 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-04-23T18:21:35.096 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-23T18:21:35.097 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at 57e0caf76fd qa/suites/fs: add quiescer to the fs suite 2024-04-23T18:21:35.098 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T18:21:35.128 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T18:21:35.130 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:21:35.131 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T18:21:35.813 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T18:21:35.882 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T18:21:35.924 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:21:35.925 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:21:35.925 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T18:21:35.967 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:21:35.970 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:21:35.970 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T18:21:36.027 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-23T18:21:36.028 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-23T18:21:36.028 DEBUG:teuthology.orchestra.run.smithi037:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-23T18:21:36.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:35 smithi172 ceph-mon[32456]: pgmap v37: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-23T18:21:36.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:36.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:35 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:36.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:35 smithi173 ceph-mon[32906]: pgmap v37: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-23T18:21:36.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:36.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:36.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:35 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:35 smithi037 ceph-mon[29404]: pgmap v37: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-23T18:21:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:35 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:37.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:36 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:37.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:36 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:37.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:36 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:36 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:36 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:36 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:37.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:36 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:37.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:36 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:37.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:36 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:21:38.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:37 smithi172 ceph-mon[32456]: pgmap v38: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:21:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:37 smithi173 ceph-mon[32906]: pgmap v38: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:21:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:37 smithi037 ceph-mon[29404]: pgmap v38: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T18:21:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 -- 172.21.15.37:0/1950546196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c1031e0 msgr2=0x7ff95c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 --2- 172.21.15.37:0/1950546196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c1031e0 0x7ff95c1035c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009ca0 tx=0x7ff94c02a9e0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 -- 172.21.15.37:0/1950546196 shutdown_connections 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 --2- 172.21.15.37:0/1950546196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c10c520 0x7ff95c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 --2- 172.21.15.37:0/1950546196 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c103b00 0x7ff95c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 --2- 172.21.15.37:0/1950546196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c1031e0 0x7ff95c1035c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 -- 172.21.15.37:0/1950546196 >> 172.21.15.37:0/1950546196 conn(0x7ff95c077fd0 msgr2=0x7ff95c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 -- 172.21.15.37:0/1950546196 shutdown_connections 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 -- 172.21.15.37:0/1950546196 wait complete. 2024-04-23T18:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.633+0000 7ff961936640 1 Processor -- start 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 -- start start 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c1031e0 0x7ff95c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c103b00 0x7ff95c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 0x7ff95c1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c112210 con 0x7ff95c10c520 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c112090 con 0x7ff95c103b00 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff961936640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c112390 con 0x7ff95c1031e0 2024-04-23T18:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 0x7ff95c1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c1031e0 0x7ff95c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 0x7ff95c1018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35452/0 (socket says 172.21.15.37:35452) 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95b7fe640 1 -- 172.21.15.37:0/3330310351 learned_addr learned my addr 172.21.15.37:0/3330310351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c103b00 0x7ff95c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95b7fe640 1 -- 172.21.15.37:0/3330310351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c1031e0 msgr2=0x7ff95c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95b7fe640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c1031e0 0x7ff95c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.634+0000 7ff95b7fe640 1 -- 172.21.15.37:0/3330310351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c103b00 msgr2=0x7ff95c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff95b7fe640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c103b00 0x7ff95c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff95b7fe640 1 -- 172.21.15.37:0/3330310351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c1a6c50 con 0x7ff95c10c520 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff95b7fe640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 0x7ff95c1018b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff95400d950 tx=0x7ff95400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff960934640 1 -- 172.21.15.37:0/3330310351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954017030 con 0x7ff95c10c520 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff960934640 1 -- 172.21.15.37:0/3330310351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff954004020 con 0x7ff95c10c520 2024-04-23T18:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c1a6f40 con 0x7ff95c10c520 2024-04-23T18:21:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff960934640 1 -- 172.21.15.37:0/3330310351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954003af0 con 0x7ff95c10c520 2024-04-23T18:21:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.635+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c1a7460 con 0x7ff95c10c520 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.637+0000 7ff960934640 1 -- 172.21.15.37:0/3330310351 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff95401a070 con 0x7ff95c10c520 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.637+0000 7ff964655740 1 Processor -- start 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 -- start start 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 0x5590f6e3ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e4bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590f6da3970 con 0x5590f6e3e620 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5590f6d404a0 con 0x5590f6e49800 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff964655740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5590f6d9f930 con 0x5590f6e3ef40 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 0x5590f6e3ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 0x5590f6e3ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35466/0 (socket says 172.21.15.37:35466) 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 -- 172.21.15.37:0/966088642 learned_addr learned my addr 172.21.15.37:0/966088642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff960934640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff9440a3f60 0x7ff9440a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95affd640 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95b7fe640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff9440a3f60 0x7ff9440a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 -- 172.21.15.37:0/966088642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 msgr2=0x5590f6e492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 -- 172.21.15.37:0/966088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 msgr2=0x5590f6e4bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e4bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff95a7fc640 1 -- 172.21.15.37:0/966088642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f6ab6210 con 0x5590f6e3e620 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.638+0000 7ff960934640 1 -- 172.21.15.37:0/3330310351 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff9540d0500 con 0x7ff95c10c520 2024-04-23T18:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 0x5590f6e3ea00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff950006aa0 tx=0x7ff950009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3904fe110974e1b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff941ffb640 1 -- 172.21.15.37:0/966088642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950018020 con 0x5590f6e3e620 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff95b7fe640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff9440a3f60 0x7ff9440a6420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cdac0 tx=0x7ff95406dc20 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff941ffb640 1 -- 172.21.15.37:0/966088642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff950010dc0 con 0x5590f6e3e620 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff941ffb640 1 -- 172.21.15.37:0/966088642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950013690 con 0x5590f6e3e620 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff964655740 1 -- 172.21.15.37:0/966088642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 msgr2=0x5590f6e3ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.639+0000 7ff964655740 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 0x5590f6e3ea00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff950006aa0 tx=0x7ff950009f70 comp rx=0 tx=0).stop 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 -- 172.21.15.37:0/966088642 shutdown_connections 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e4bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 --2- 172.21.15.37:0/966088642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3e620 0x5590f6e3ea00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 -- 172.21.15.37:0/966088642 >> 172.21.15.37:0/966088642 conn(0x5590f6e3ab30 msgr2=0x5590f6e3af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 -- 172.21.15.37:0/966088642 shutdown_connections 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 -- 172.21.15.37:0/966088642 wait complete. 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 -- start start 2024-04-23T18:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 Processor -- start 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3e620 0x5590f6e490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.640+0000 7ff964655740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff964655740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e46900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff964655740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590f6e4c430 con 0x5590f6e3ef40 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff964655740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5590f6d3e0b0 con 0x5590f6e49800 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff964655740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5590f6e4c5b0 con 0x5590f6e3e620 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff964655740 10 client.0 _ll_register_callbacks cb 0x5590f6d2bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35470/0 (socket says 172.21.15.37:35470) 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 -- 172.21.15.37:0/2640928090 learned_addr learned my addr 172.21.15.37:0/2640928090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3e620 0x5590f6e490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e46900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 -- 172.21.15.37:0/2640928090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3e620 msgr2=0x5590f6e490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3e620 0x5590f6e490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 -- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 msgr2=0x5590f6e46900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e46900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 -- 172.21.15.37:0/2640928090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f6b82620 con 0x5590f6e3ef40 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e46900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff95affd640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e463c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff94c035000 tx=0x7ff94c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381bb9fde8317a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff93affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff93affd640 1 -- 172.21.15.37:0/2640928090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c048060 con 0x5590f6e3ef40 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff93affd640 1 -- 172.21.15.37:0/2640928090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff94c02e590 con 0x5590f6e3ef40 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.641+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff950010dc0 con 0x5590f6e3ef40 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff93affd640 1 -- 172.21.15.37:0/2640928090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff94c033040 con 0x5590f6e3ef40 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590f6e58310 con 0x5590f6e3ef40 2024-04-23T18:21:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff93affd640 1 -- 172.21.15.37:0/2640928090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff950010dc0 con 0x5590f6e3ef40 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff93affd640 1 -- 172.21.15.37:0/2640928090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff94c02f640 con 0x5590f6e3ef40 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 20 client.15684 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6e59890 0x5590f6e5bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 4 client.15684 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590f6db1390 con 0x5590f6e59890 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.642+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6e59890 0x5590f6e5bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.643+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6e59890 0x5590f6e5bcb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4529f752f7b24489 server_cookie=6511860caeaad507 in_seq=0 out_seq=0 2024-04-23T18:21:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.643+0000 7ff93affd640 10 client.15684 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff93affd640 1 -- 172.21.15.37:0/2640928090 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590f6db1390 con 0x5590f6e59890 2024-04-23T18:21:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff93affd640 10 client.15684 handle_command_reply: tid=0 2024-04-23T18:21:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 client.15684 shutdown 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 msgr2=0x5590f6e463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e463c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff94c035000 tx=0x7ff94c003ba0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 shutdown_connections 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6e59890 0x5590f6e5bcb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e49800 0x5590f6e46900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e463c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff93affd640 0 client.15684 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 --2- 172.21.15.37:0/2640928090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3e620 0x5590f6e490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 >> 172.21.15.37:0/2640928090 conn(0x5590f6e3ab30 msgr2=0x5590f6e40860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 shutdown_connections 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 1 -- 172.21.15.37:0/2640928090 wait complete. 2024-04-23T18:21:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.644+0000 7ff964655740 20 client.15684 trim_cache size 0 max 16384 2024-04-23T18:21:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 Processor -- start 2024-04-23T18:21:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 -- start start 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e48c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e49800 0x5590f6e615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1b26f0 con 0x5590f6e3ef40 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1affc0 con 0x5590f6e3e620 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff964655740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ade10 con 0x5590f6e49800 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff95affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff95affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e48c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35472/0 (socket says 172.21.15.37:35472) 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.659+0000 7ff95affd640 1 -- 172.21.15.37:0/2204832557 learned_addr learned my addr 172.21.15.37:0/2204832557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e49800 0x5590f6e615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95affd640 1 -- 172.21.15.37:0/2204832557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e49800 msgr2=0x5590f6e615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95affd640 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e49800 0x5590f6e615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95affd640 1 -- 172.21.15.37:0/2204832557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 msgr2=0x5590f6e552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95affd640 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95affd640 1 -- 172.21.15.37:0/2204832557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f6e3f6b0 con 0x5590f6e3ef40 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff95affd640 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e48c00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff94c02fc00 tx=0x7ff94c02a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4f8719b5201ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff9417fa640 1 -- 172.21.15.37:0/2204832557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c048040 con 0x5590f6e3ef40 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff9417fa640 1 -- 172.21.15.37:0/2204832557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94c02e840 con 0x5590f6e3ef40 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff9417fa640 1 -- 172.21.15.37:0/2204832557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c004330 con 0x5590f6e3ef40 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff964655740 1 -- 172.21.15.37:0/2204832557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 msgr2=0x5590f6e48c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.660+0000 7ff964655740 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e48c00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff94c02fc00 tx=0x7ff94c02a9e0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- 172.21.15.37:0/2204832557 shutdown_connections 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e49800 0x5590f6e615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e3ef40 0x5590f6e48c00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 --2- 172.21.15.37:0/2204832557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- 172.21.15.37:0/2204832557 >> 172.21.15.37:0/2204832557 conn(0x5590f6e3ab30 msgr2=0x5590f6e46380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- 172.21.15.37:0/2204832557 shutdown_connections 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- 172.21.15.37:0/2204832557 wait complete. 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- start start 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 Processor -- start 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 0x5590f6e4d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ae480 con 0x5590f6e49800 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1af8f0 con 0x5590f6e3e620 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1aeb50 con 0x5590f6e3ef40 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.661+0000 7ff964655740 10 client.0 _ll_register_callbacks cb 0x7ff94c02ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 0x5590f6e4d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 0x5590f6e4d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35474/0 (socket says 172.21.15.37:35474) 2024-04-23T18:21:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 -- 172.21.15.37:0/2853676588 learned_addr learned my addr 172.21.15.37:0/2853676588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95affd640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 -- 172.21.15.37:0/2853676588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 msgr2=0x5590f6e4cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e4cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 -- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 msgr2=0x5590f6e518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 -- 172.21.15.37:0/2853676588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f6e4d800 con 0x5590f6e49800 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff95a7fc640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 0x5590f6e4d2c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff950009fb0 tx=0x7ff950009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb147cb98ff44c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff93affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff93affd640 1 -- 172.21.15.37:0/2853676588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950018040 con 0x5590f6e49800 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff93affd640 1 -- 172.21.15.37:0/2853676588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff95002cae0 con 0x5590f6e49800 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94c02e840 con 0x5590f6e49800 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff93affd640 1 -- 172.21.15.37:0/2853676588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff95002c3d0 con 0x5590f6e49800 2024-04-23T18:21:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff964655740 10 client.15690 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.662+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590f6ebfb50 con 0x5590f6e49800 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff93affd640 1 -- 172.21.15.37:0/2853676588 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94c02e840 con 0x5590f6e49800 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff93affd640 1 -- 172.21.15.37:0/2853676588 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff95002cce0 con 0x5590f6e49800 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff964655740 10 client.15690 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff964655740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff964655740 20 client.15690 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff964655740 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6ec01c0 0x5590f6ec25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff964655740 4 client.15690 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590f6dab200 con 0x5590f6ec01c0 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff95affd640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6ec01c0 0x5590f6ec25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff93affd640 10 client.15690 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.663+0000 7ff95affd640 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6ec01c0 0x5590f6ec25e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37546daa32bb40d server_cookie=f303e2c3bc2da411 in_seq=0 out_seq=0 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff93affd640 1 -- 172.21.15.37:0/2853676588 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590f6dab200 con 0x5590f6ec01c0 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff93affd640 10 client.15690 handle_command_reply: tid=0 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 client.15690 shutdown 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 msgr2=0x5590f6e4d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 0x5590f6e4d2c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff950009fb0 tx=0x7ff950009420 comp rx=0 tx=0).stop 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 shutdown_connections 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5590f6ec01c0 0x5590f6ec25e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f6e49800 0x5590f6e4d2c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff93affd640 0 client.15690 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590f6e3ef40 0x5590f6e4cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 --2- 172.21.15.37:0/2853676588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5590f6e3e620 0x5590f6e518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 >> 172.21.15.37:0/2853676588 conn(0x5590f6e3ab30 msgr2=0x5590f6e41210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 shutdown_connections 2024-04-23T18:21:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 1 -- 172.21.15.37:0/2853676588 wait complete. 2024-04-23T18:21:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.668+0000 7ff964655740 20 client.15690 trim_cache size 0 max 16384 2024-04-23T18:21:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:21:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:21:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:21:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff9440a3f60 msgr2=0x7ff9440a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff9440a3f60 0x7ff9440a6420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cdac0 tx=0x7ff95406dc20 comp rx=0 tx=0).stop 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 msgr2=0x7ff95c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 0x7ff95c1018b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff95400d950 tx=0x7ff95400dd50 comp rx=0 tx=0).stop 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 shutdown_connections 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff9440a3f60 0x7ff9440a6420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95c10c520 0x7ff95c1018b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c103b00 0x7ff95c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 --2- 172.21.15.37:0/3330310351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c1031e0 0x7ff95c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 >> 172.21.15.37:0/3330310351 conn(0x7ff95c077fd0 msgr2=0x7ff95c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 shutdown_connections 2024-04-23T18:21:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:38.669+0000 7ff961936640 1 -- 172.21.15.37:0/3330310351 wait complete. 2024-04-23T18:21:39.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:39 smithi172 ceph-mon[32456]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-23T18:21:39.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:39 smithi172 ceph-mon[32456]: Cluster is now healthy 2024-04-23T18:21:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:39 smithi173 ceph-mon[32906]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-23T18:21:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:39 smithi173 ceph-mon[32906]: Cluster is now healthy 2024-04-23T18:21:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:39 smithi037 ceph-mon[29404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-23T18:21:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:39 smithi037 ceph-mon[29404]: Cluster is now healthy 2024-04-23T18:21:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:40 smithi172 ceph-mon[32456]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:21:40.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:40 smithi173 ceph-mon[32906]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:21:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:40 smithi037 ceph-mon[29404]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:21:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:42 smithi172 ceph-mon[32456]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 106 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:21:42.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:42 smithi173 ceph-mon[32906]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 106 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:21:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:42 smithi037 ceph-mon[29404]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 106 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-23T18:21:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:44 smithi172 ceph-mon[32456]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 770 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:44.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:44 smithi173 ceph-mon[32906]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 770 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:44 smithi037 ceph-mon[29404]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 770 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:46 smithi172 ceph-mon[32456]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:46.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:46 smithi173 ceph-mon[32906]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:46 smithi037 ceph-mon[29404]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:48 smithi172 ceph-mon[32456]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:48.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:48 smithi173 ceph-mon[32906]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:48 smithi037 ceph-mon[29404]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:50.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:50 smithi172 ceph-mon[32456]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:50.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:50 smithi173 ceph-mon[32906]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:50 smithi037 ceph-mon[29404]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:21:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 -- 172.21.15.37:0/2923764810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb98102b70 msgr2=0x7feb98106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/2923764810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb98102b70 0x7feb98106bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009900 tx=0x7feb8c02a410 comp rx=0 tx=0).stop 2024-04-23T18:21:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 -- 172.21.15.37:0/2923764810 shutdown_connections 2024-04-23T18:21:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/2923764810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb98107290 0x7feb9810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/2923764810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb98102b70 0x7feb98106bb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/2923764810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb981021c0 0x7feb981025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 -- 172.21.15.37:0/2923764810 >> 172.21.15.37:0/2923764810 conn(0x7feb980fda20 msgr2=0x7feb980ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 -- 172.21.15.37:0/2923764810 shutdown_connections 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 -- 172.21.15.37:0/2923764810 wait complete. 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 Processor -- start 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.770+0000 7feb9fbbb640 1 -- start start 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9fbbb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb981021c0 0x7feb98198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9fbbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb98107290 0x7feb98198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9fbbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 0x7feb9819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9fbbb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb981121b0 con 0x7feb9819cf30 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9fbbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb98112030 con 0x7feb98107290 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9fbbb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb98112330 con 0x7feb981021c0 2024-04-23T18:21:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 0x7feb9819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 0x7feb9819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33706/0 (socket says 172.21.15.37:33706) 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 -- 172.21.15.37:0/1656438887 learned_addr learned my addr 172.21.15.37:0/1656438887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9d156640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb98107290 0x7feb98198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9d957640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb981021c0 0x7feb98198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 -- 172.21.15.37:0/1656438887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb981021c0 msgr2=0x7feb98198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb981021c0 0x7feb98198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 -- 172.21.15.37:0/1656438887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb98107290 msgr2=0x7feb98198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb98107290 0x7feb98198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 -- 172.21.15.37:0/1656438887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9819da20 con 0x7feb9819cf30 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.771+0000 7feb9e158640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 0x7feb9819d3e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feb8800cce0 tx=0x7feb8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.772+0000 7feb96ffd640 1 -- 172.21.15.37:0/1656438887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8800f450 con 0x7feb9819cf30 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.772+0000 7feb96ffd640 1 -- 172.21.15.37:0/1656438887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8800faa0 con 0x7feb9819cf30 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.772+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9819dd10 con 0x7feb9819cf30 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.772+0000 7feb96ffd640 1 -- 172.21.15.37:0/1656438887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88018b70 con 0x7feb9819cf30 2024-04-23T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.772+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb981a5480 con 0x7feb9819cf30 2024-04-23T18:21:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.774+0000 7feb96ffd640 1 -- 172.21.15.37:0/1656438887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7feb88018d10 con 0x7feb9819cf30 2024-04-23T18:21:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 Processor -- start 2024-04-23T18:21:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 -- start start 2024-04-23T18:21:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8c948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5593d8bf9970 con 0x5593d8ca3c10 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593d8b964a0 con 0x5593d8c94500 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feba29ab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5593d8bf5930 con 0x5593d8c94e20 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feb96ffd640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb800a3ee0 0x7feb800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33718/0 (socket says 172.21.15.37:33718) 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feb9d156640 1 -- 172.21.15.37:0/4107242287 learned_addr learned my addr 172.21.15.37:0/4107242287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feb9d957640 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.775+0000 7feb9e158640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb800a3ee0 0x7feb800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8c948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb96ffd640 1 -- 172.21.15.37:0/1656438887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb880d0e80 con 0x7feb9819cf30 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 -- 172.21.15.37:0/4107242287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 msgr2=0x5593d8ca36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 -- 172.21.15.37:0/4107242287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 msgr2=0x5593d8c948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8c948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 -- 172.21.15.37:0/4107242287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d890c210 con 0x5593d8ca3c10 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9d156640 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca6000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feb8c033000 tx=0x7feb8c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77a7745d47267b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb9e158640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb800a3ee0 0x7feb800a63a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb880ce440 tx=0x7feb8806c440 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb7ffff640 1 -- 172.21.15.37:0/4107242287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c035070 con 0x5593d8ca3c10 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb7ffff640 1 -- 172.21.15.37:0/4107242287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c031620 con 0x5593d8ca3c10 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.776+0000 7feb7ffff640 1 -- 172.21.15.37:0/4107242287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c003c00 con 0x5593d8ca3c10 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 -- 172.21.15.37:0/4107242287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 msgr2=0x5593d8ca6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca6000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feb8c033000 tx=0x7feb8c005780 comp rx=0 tx=0).stop 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 -- 172.21.15.37:0/4107242287 shutdown_connections 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca6000 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 --2- 172.21.15.37:0/4107242287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8c948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 -- 172.21.15.37:0/4107242287 >> 172.21.15.37:0/4107242287 conn(0x5593d8c90b30 msgr2=0x5593d8c90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 -- 172.21.15.37:0/4107242287 shutdown_connections 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 -- 172.21.15.37:0/4107242287 wait complete. 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.777+0000 7feba29ab740 1 -- start start 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 Processor -- start 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8ca3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5593d8ca67e0 con 0x5593d8ca3c10 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593d8b940b0 con 0x5593d8c94500 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5593d8ca6960 con 0x5593d8c94e20 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feba29ab740 10 client.0 _ll_register_callbacks cb 0x5593d8b81d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33730/0 (socket says 172.21.15.37:33730) 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feb9d156640 1 -- 172.21.15.37:0/1195842498 learned_addr learned my addr 172.21.15.37:0/1195842498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feb9d957640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feb9d156640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8ca3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.778+0000 7feb9d156640 1 -- 172.21.15.37:0/1195842498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 msgr2=0x5593d8ca0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb9d156640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb9d156640 1 -- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 msgr2=0x5593d8ca3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb9d156640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8ca3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb9d156640 1 -- 172.21.15.37:0/1195842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d89d8620 con 0x5593d8ca3c10 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb9d156640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca0dc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009900 tx=0x7feb8c0037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaf1bf8b9f151c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb7d7fa640 1 -- 172.21.15.37:0/1195842498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c035040 con 0x5593d8ca3c10 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c031620 con 0x5593d8ca3c10 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb7d7fa640 1 -- 172.21.15.37:0/1195842498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb8c045070 con 0x5593d8ca3c10 2024-04-23T18:21:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb7d7fa640 1 -- 172.21.15.37:0/1195842498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb8c05b050 con 0x5593d8ca3c10 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feba29ab740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593d8cb2790 con 0x5593d8ca3c10 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb7d7fa640 1 -- 172.21.15.37:0/1195842498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c031620 con 0x5593d8ca3c10 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.779+0000 7feb7d7fa640 1 -- 172.21.15.37:0/1195842498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feb8c03d450 con 0x5593d8ca3c10 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feba29ab740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feba29ab740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feba29ab740 20 client.15702 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feba29ab740 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8cb3ae0 0x5593d8cb5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feba29ab740 4 client.15702 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593d8c07390 con 0x5593d8cb3ae0 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feb9d957640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8cb3ae0 0x5593d8cb5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feb7d7fa640 10 client.15702 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.780+0000 7feb9d957640 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8cb3ae0 0x5593d8cb5ed0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f458e0b21cc5a26 server_cookie=88359d074edc5048 in_seq=0 out_seq=0 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.781+0000 7feb7d7fa640 1 -- 172.21.15.37:0/1195842498 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593d8c07390 con 0x5593d8cb3ae0 2024-04-23T18:21:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.781+0000 7feb7d7fa640 10 client.15702 handle_command_reply: tid=0 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.781+0000 7feba29ab740 1 client.15702 shutdown 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.781+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 msgr2=0x5593d8ca0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.781+0000 7feba29ab740 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca0dc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009900 tx=0x7feb8c0037e0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 shutdown_connections 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8cb3ae0 0x5593d8cb5ed0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca0dc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 --2- 172.21.15.37:0/1195842498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94500 0x5593d8ca3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feb7d7fa640 0 client.15702 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 >> 172.21.15.37:0/1195842498 conn(0x5593d8c90b30 msgr2=0x5593d8c969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 shutdown_connections 2024-04-23T18:21:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 1 -- 172.21.15.37:0/1195842498 wait complete. 2024-04-23T18:21:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.782+0000 7feba29ab740 20 client.15702 trim_cache size 0 max 16384 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 Processor -- start 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 -- start start 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 0x5593d8caf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8ca3c10 0x5593d8cbb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb981ae0a0 con 0x5593d8c94500 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb981ab970 con 0x5593d8ca3c10 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feba29ab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a97e0 con 0x5593d8c94e20 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 0x5593d8caf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 0x5593d8caf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33746/0 (socket says 172.21.15.37:33746) 2024-04-23T18:21:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 -- 172.21.15.37:0/3139989274 learned_addr learned my addr 172.21.15.37:0/3139989274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d957640 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 -- 172.21.15.37:0/3139989274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 msgr2=0x5593d8ca30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 -- 172.21.15.37:0/3139989274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8ca3c10 msgr2=0x5593d8cbb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.797+0000 7feb9d156640 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8ca3c10 0x5593d8cbb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feb9d156640 1 -- 172.21.15.37:0/3139989274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d8c90960 con 0x5593d8c94500 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feb9d156640 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 0x5593d8caf6e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009850 tx=0x7feb8c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227d84feb35753be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feb7f7fe640 1 -- 172.21.15.37:0/3139989274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c035070 con 0x5593d8c94500 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feb7f7fe640 1 -- 172.21.15.37:0/3139989274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c032e30 con 0x5593d8c94500 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feb7f7fe640 1 -- 172.21.15.37:0/3139989274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c0319e0 con 0x5593d8c94500 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feba29ab740 1 -- 172.21.15.37:0/3139989274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 msgr2=0x5593d8caf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feba29ab740 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 0x5593d8caf6e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009850 tx=0x7feb8c031400 comp rx=0 tx=0).stop 2024-04-23T18:21:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feba29ab740 1 -- 172.21.15.37:0/3139989274 shutdown_connections 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feba29ab740 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8ca3c10 0x5593d8cbb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feba29ab740 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94e20 0x5593d8ca30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.798+0000 7feba29ab740 1 --2- 172.21.15.37:0/3139989274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8c94500 0x5593d8caf6e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- 172.21.15.37:0/3139989274 >> 172.21.15.37:0/3139989274 conn(0x5593d8c90b30 msgr2=0x5593d8c9bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- 172.21.15.37:0/3139989274 shutdown_connections 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- 172.21.15.37:0/3139989274 wait complete. 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- start start 2024-04-23T18:21:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 Processor -- start 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94500 0x5593d8ca75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94e20 0x5593d8ca7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb981a9e30 con 0x5593d8ca3c10 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb981ab2a0 con 0x5593d8c94e20 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb981aa500 con 0x5593d8c94500 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feba29ab740 10 client.0 _ll_register_callbacks cb 0x7feb8c032ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.799+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33752/0 (socket says 172.21.15.37:33752) 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 -- 172.21.15.37:0/53420572 learned_addr learned my addr 172.21.15.37:0/53420572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d957640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94e20 0x5593d8ca7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94500 0x5593d8ca75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 -- 172.21.15.37:0/53420572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94500 msgr2=0x5593d8ca75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94500 0x5593d8ca75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 -- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94e20 msgr2=0x5593d8ca7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94e20 0x5593d8ca7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 -- 172.21.15.37:0/53420572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d8d18dd0 con 0x5593d8ca3c10 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb9d156640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca8050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7feb8c03d940 tx=0x7feb8c005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdbc8846f26824d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:51.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb7d7fa640 1 -- 172.21.15.37:0/53420572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c035070 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb7d7fa640 1 -- 172.21.15.37:0/53420572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb8c005480 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c032e30 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feb7d7fa640 1 -- 172.21.15.37:0/53420572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb8c0046c0 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feba29ab740 10 client.15708 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.800+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593d8d19ff0 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feb7d7fa640 1 -- 172.21.15.37:0/53420572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c032e30 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feb7d7fa640 1 -- 172.21.15.37:0/53420572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7feb8c04a790 con 0x5593d8ca3c10 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feba29ab740 10 client.15708 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feba29ab740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feba29ab740 20 client.15708 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feba29ab740 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8d1a4c0 0x5593d8d1c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feba29ab740 4 client.15708 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:21:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593d8c01200 con 0x5593d8d1a4c0 2024-04-23T18:21:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.801+0000 7feb9d957640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8d1a4c0 0x5593d8d1c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.802+0000 7feb9d957640 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8d1a4c0 0x5593d8d1c8b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7bcab67485514e server_cookie=9462563fb6921240 in_seq=0 out_seq=0 2024-04-23T18:21:51.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.802+0000 7feb7d7fa640 10 client.15708 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.805+0000 7feb7d7fa640 1 -- 172.21.15.37:0/53420572 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593d8c01200 con 0x5593d8d1a4c0 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.805+0000 7feb7d7fa640 10 client.15708 handle_command_reply: tid=0 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.805+0000 7feba29ab740 1 client.15708 shutdown 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 msgr2=0x5593d8ca8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca8050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7feb8c03d940 tx=0x7feb8c005950 comp rx=0 tx=0).stop 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 shutdown_connections 2024-04-23T18:21:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5593d8d1a4c0 0x5593d8d1c8b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593d8ca3c10 0x5593d8ca8050 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d8c94e20 0x5593d8ca7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 --2- 172.21.15.37:0/53420572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5593d8c94500 0x5593d8ca75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb7d7fa640 0 client.15708 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 >> 172.21.15.37:0/53420572 conn(0x5593d8c90b30 msgr2=0x5593d8c8f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 shutdown_connections 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 1 -- 172.21.15.37:0/53420572 wait complete. 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feba29ab740 20 client.15708 trim_cache size 0 max 16384 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:21:51.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:21:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:21:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:21:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb800a3ee0 msgr2=0x7feb800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb800a3ee0 0x7feb800a63a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb880ce440 tx=0x7feb8806c440 comp rx=0 tx=0).stop 2024-04-23T18:21:51.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 msgr2=0x7feb9819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 0x7feb9819d3e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feb8800cce0 tx=0x7feb8800a620 comp rx=0 tx=0).stop 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 shutdown_connections 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb800a3ee0 0x7feb800a63a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb9819cf30 0x7feb9819d3e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb98107290 0x7feb98198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 --2- 172.21.15.37:0/1656438887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb981021c0 0x7feb98198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.806+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 >> 172.21.15.37:0/1656438887 conn(0x7feb980fda20 msgr2=0x7feb980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.807+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 shutdown_connections 2024-04-23T18:21:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:21:51.807+0000 7feb9fbbb640 1 -- 172.21.15.37:0/1656438887 wait complete. 2024-04-23T18:21:52.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:52 smithi172 ceph-mon[32456]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:21:52.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:52 smithi173 ceph-mon[32906]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:21:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:52 smithi037 ceph-mon[29404]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T18:21:54.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:54 smithi172 ceph-mon[32456]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:21:54.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:54 smithi173 ceph-mon[32906]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:21:54.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:54 smithi037 ceph-mon[29404]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T18:21:55.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:21:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/4136317943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd410e910 msgr2=0x7f0fd4110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/4136317943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd410e910 0x7f0fd4110d00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009ca0 tx=0x7f0fcc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/4136317943 shutdown_connections 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/4136317943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd410e910 0x7f0fd4110d00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/4136317943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fd4101680 0x7f0fd410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/4136317943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd4100d60 0x7f0fd4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/4136317943 >> 172.21.15.37:0/4136317943 conn(0x7f0fd4077f50 msgr2=0x7f0fd4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/4136317943 shutdown_connections 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.308+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/4136317943 wait complete. 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 Processor -- start 2024-04-23T18:21:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 -- start start 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 0x7f0fd41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fd4101680 0x7f0fd41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd410e910 0x7f0fd419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41143d0 con 0x7f0fd4100d60 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4114250 con 0x7f0fd410e910 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fdb3ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4114550 con 0x7f0fd4101680 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 0x7f0fd41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 0x7f0fd41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57006/0 (socket says 172.21.15.37:57006) 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 -- 172.21.15.37:0/3672097264 learned_addr learned my addr 172.21.15.37:0/3672097264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd8948640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fd4101680 0x7f0fd41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd994a640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd410e910 0x7f0fd419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 -- 172.21.15.37:0/3672097264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fd4101680 msgr2=0x7f0fd41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fd4101680 0x7f0fd41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 -- 172.21.15.37:0/3672097264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd410e910 msgr2=0x7f0fd419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd410e910 0x7f0fd419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 -- 172.21.15.37:0/3672097264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd419b010 con 0x7f0fd4100d60 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd994a640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd410e910 0x7f0fd419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.309+0000 7f0fd9149640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 0x7f0fd41a0690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e9f0 tx=0x7f0fd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.310+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000cd80 con 0x7f0fd4100d60 2024-04-23T18:21:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.310+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fd00108e0 con 0x7f0fd4100d60 2024-04-23T18:21:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.310+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0018980 con 0x7f0fd4100d60 2024-04-23T18:21:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.310+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd419b300 con 0x7f0fd4100d60 2024-04-23T18:21:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.310+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41a7610 con 0x7f0fd4100d60 2024-04-23T18:21:55.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.311+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd40698e0 con 0x7f0fd4100d60 2024-04-23T18:21:55.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.312+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f0fd0010430 con 0x7f0fd4100d60 2024-04-23T18:21:55.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.313+0000 7f0fca7fc640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0fc00a3e40 0x7f0fc00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.313+0000 7f0fd8948640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0fc00a3e40 0x7f0fc00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.313+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0fd00d1360 con 0x7f0fd4100d60 2024-04-23T18:21:55.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.313+0000 7f0fd8948640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0fc00a3e40 0x7f0fc00a6300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fd419bfa0 tx=0x7f0fc400b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.316+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fd009d740 con 0x7f0fd4100d60 2024-04-23T18:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.544+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fd419bc80 con 0x7f0fd4100d60 2024-04-23T18:21:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.545+0000 7f0fca7fc640 1 -- 172.21.15.37:0/3672097264 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11878 (secure 0 0 0) 0x7f0fd009d040 con 0x7f0fd4100d60 2024-04-23T18:21:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:21:55.538 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:20:58.538288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:21:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.546+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0fc00a3e40 msgr2=0x7f0fc00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.546+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0fc00a3e40 0x7f0fc00a6300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fd419bfa0 tx=0x7f0fc400b390 comp rx=0 tx=0).stop 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.546+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 msgr2=0x7f0fd41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.546+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 0x7f0fd41a0690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e9f0 tx=0x7f0fd000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 shutdown_connections 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0fc00a3e40 0x7f0fc00a6300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fd410e910 0x7f0fd419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fd4101680 0x7f0fd41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 --2- 172.21.15.37:0/3672097264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0fd4100d60 0x7f0fd41a0690 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 >> 172.21.15.37:0/3672097264 conn(0x7f0fd4077f50 msgr2=0x7f0fd410fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 shutdown_connections 2024-04-23T18:21:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.547+0000 7f0fdb3ad640 1 -- 172.21.15.37:0/3672097264 wait complete. 2024-04-23T18:21:55.551 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 1d7c0ffc --release --await' 2024-04-23T18:21:55.551 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id 1d7c0ffc --release --await 2024-04-23T18:21:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 -- 172.21.15.37:0/1645149172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 msgr2=0x7f41b810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 --2- 172.21.15.37:0/1645149172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b810ee40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009ca0 tx=0x7f41b002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 -- 172.21.15.37:0/1645149172 shutdown_connections 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 --2- 172.21.15.37:0/1645149172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b810ee40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 --2- 172.21.15.37:0/1645149172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b806b1f0 0x7f41b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 --2- 172.21.15.37:0/1645149172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41b806a840 0x7f41b806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 -- 172.21.15.37:0/1645149172 >> 172.21.15.37:0/1645149172 conn(0x7f41b8077eb0 msgr2=0x7f41b80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 -- 172.21.15.37:0/1645149172 shutdown_connections 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.666+0000 7f41bd708640 1 -- 172.21.15.37:0/1645149172 wait complete. 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 Processor -- start 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 -- start start 2024-04-23T18:21:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41b806a840 0x7f41b81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b819a730 0x7f41b819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81121d0 con 0x7f41b8107270 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8112050 con 0x7f41b819a730 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.667+0000 7f41bd708640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8112350 con 0x7f41b806a840 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b81a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57012/0 (socket says 172.21.15.37:57012) 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 -- 172.21.15.37:0/1314042918 learned_addr learned my addr 172.21.15.37:0/1314042918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41b806a840 0x7f41b81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b77fe640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b819a730 0x7f41b819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 -- 172.21.15.37:0/1314042918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41b806a840 msgr2=0x7f41b81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41b806a840 0x7f41b81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 -- 172.21.15.37:0/1314042918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b819a730 msgr2=0x7f41b819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b819a730 0x7f41b819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 -- 172.21.15.37:0/1314042918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b819b330 con 0x7f41b8107270 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41b67fc640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b81a0b60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f41a000d770 tx=0x7f41a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41affff640 1 -- 172.21.15.37:0/1314042918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x7f41b8107270 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41affff640 1 -- 172.21.15.37:0/1314042918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a0009c30 con 0x7f41b8107270 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b81a73c0 con 0x7f41b8107270 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41affff640 1 -- 172.21.15.37:0/1314042918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000f620 con 0x7f41b8107270 2024-04-23T18:21:55.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.668+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a77c0 con 0x7f41b8107270 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.670+0000 7f41affff640 1 -- 172.21.15.37:0/1314042918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f41a001a070 con 0x7f41b8107270 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 Processor -- start 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 -- start start 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7def9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df03f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df06870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bd7de5e6d0 con 0x55bd7df04480 2024-04-23T18:21:55.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bd7ddfb1a0 con 0x55bd7def9490 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41c041e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bd7de5a690 con 0x55bd7def9db0 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41affff640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41a40a3e40 0x7f41a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41b67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df06870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41b67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df06870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57028/0 (socket says 172.21.15.37:57028) 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41b67fc640 1 -- 172.21.15.37:0/2761968295 learned_addr learned my addr 172.21.15.37:0/2761968295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df03f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41b77fe640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41a40a3e40 0x7f41a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41affff640 1 -- 172.21.15.37:0/1314042918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f41a00d04c0 con 0x7f41b8107270 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.671+0000 7f41b77fe640 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7def9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b67fc640 1 -- 172.21.15.37:0/2761968295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 msgr2=0x55bd7df03f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df03f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b67fc640 1 -- 172.21.15.37:0/2761968295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 msgr2=0x55bd7def9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7def9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b67fc640 1 -- 172.21.15.37:0/2761968295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7de055b0 con 0x55bd7df04480 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b77fe640 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7def9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df06870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f41a009c240 tx=0x7f41a000df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a1dc8ed6aa674b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41b77fe640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41a40a3e40 0x7f41a40a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f41b002f500 tx=0x7f41b002f310 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41ad7fa640 1 -- 172.21.15.37:0/2761968295 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0021040 con 0x55bd7df04480 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41ad7fa640 1 -- 172.21.15.37:0/2761968295 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a0009de0 con 0x55bd7df04480 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.672+0000 7f41ad7fa640 1 -- 172.21.15.37:0/2761968295 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a009c770 con 0x55bd7df04480 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- 172.21.15.37:0/2761968295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 msgr2=0x55bd7df06870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df06870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f41a009c240 tx=0x7f41a000df10 comp rx=0 tx=0).stop 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- 172.21.15.37:0/2761968295 shutdown_connections 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df06870 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df03f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- 172.21.15.37:0/2761968295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7def9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- 172.21.15.37:0/2761968295 >> 172.21.15.37:0/2761968295 conn(0x55bd7def57b0 msgr2=0x55bd7def5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- 172.21.15.37:0/2761968295 shutdown_connections 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- 172.21.15.37:0/2761968295 wait complete. 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- start start 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 Processor -- start 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9490 0x55bd7df03ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 0x55bd7df011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7df04480 0x55bd7df01920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bd7df072b0 con 0x55bd7def9db0 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bd7ddf8b20 con 0x55bd7df04480 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bd7df07430 con 0x55bd7def9490 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41c041e740 10 client.0 _ll_register_callbacks cb 0x55bd7ddf8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:55.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.673+0000 7f41b67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 0x55bd7df011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 0x55bd7df011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57038/0 (socket says 172.21.15.37:57038) 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 -- 172.21.15.37:0/2462528795 learned_addr learned my addr 172.21.15.37:0/2462528795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9490 0x55bd7df03ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b77fe640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7df04480 0x55bd7df01920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 -- 172.21.15.37:0/2462528795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9490 msgr2=0x55bd7df03ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9490 0x55bd7df03ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 -- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7df04480 msgr2=0x55bd7df01920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7df04480 0x55bd7df01920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 -- 172.21.15.37:0/2462528795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7db70920 con 0x55bd7def9db0 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b77fe640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7df04480 0x55bd7df01920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 0x55bd7df011e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ce200 tx=0x7f41a00a0720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3161e6d35f6ba997 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f419e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:55.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f419e7fc640 1 -- 172.21.15.37:0/2462528795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0021040 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f419e7fc640 1 -- 172.21.15.37:0/2462528795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f41a00cdf70 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a0009de0 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f419e7fc640 1 -- 172.21.15.37:0/2462528795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41a009c2b0 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41c041e740 10 client.15726 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd7dc3cd30 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.674+0000 7f419e7fc640 1 -- 172.21.15.37:0/2462528795 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a0009de0 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f419e7fc640 1 -- 172.21.15.37:0/2462528795 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f41a0098000 con 0x55bd7def9db0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41c041e740 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41c041e740 10 client.15726 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41c041e740 20 client.15726 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41c041e740 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df144c0 0x55bd7df168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41c041e740 4 client.15726 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd7de1c500 con 0x55bd7df144c0 2024-04-23T18:21:55.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df144c0 0x55bd7df168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f419e7fc640 10 client.15726 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.675+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df144c0 0x55bd7df168b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e942f2d12f9ee4d1 server_cookie=83afa6dd732be726 in_seq=0 out_seq=0 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f419e7fc640 1 -- 172.21.15.37:0/2462528795 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd7de1c500 con 0x55bd7df144c0 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f419e7fc640 10 client.15726 handle_command_reply: tid=0 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 client.15726 shutdown 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 msgr2=0x55bd7df011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 0x55bd7df011e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ce200 tx=0x7f41a00a0720 comp rx=0 tx=0).stop 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 shutdown_connections 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df144c0 0x55bd7df168b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7df04480 0x55bd7df01920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9db0 0x55bd7df011e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f419e7fc640 0 client.15726 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 --2- 172.21.15.37:0/2462528795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9490 0x55bd7df03ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 >> 172.21.15.37:0/2462528795 conn(0x55bd7def57b0 msgr2=0x55bd7defb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 shutdown_connections 2024-04-23T18:21:55.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.676+0000 7f41c041e740 1 -- 172.21.15.37:0/2462528795 wait complete. 2024-04-23T18:21:55.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.677+0000 7f41c041e740 20 client.15726 trim_cache size 0 max 16384 2024-04-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 Processor -- start 2024-04-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 -- start start 2024-04-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 0x55bd7defc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9db0 0x55bd7df00160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7df04480 0x55bd7df03a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b2710 con 0x55bd7def9490 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81affe0 con 0x55bd7def9db0 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.694+0000 7f41c041e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81ade30 con 0x55bd7df04480 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 0x55bd7defc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 0x55bd7defc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57054/0 (socket says 172.21.15.37:57054) 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 -- 172.21.15.37:0/2157080135 learned_addr learned my addr 172.21.15.37:0/2157080135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b77fe640 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7df04480 0x55bd7df03a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 -- 172.21.15.37:0/2157080135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7df04480 msgr2=0x55bd7df03a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7df04480 0x55bd7df03a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9db0 0x55bd7df00160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 -- 172.21.15.37:0/2157080135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9db0 msgr2=0x55bd7df00160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9db0 0x55bd7df00160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 -- 172.21.15.37:0/2157080135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7df01c90 con 0x55bd7def9490 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b67fc640 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9db0 0x55bd7df00160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 0x55bd7defc750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f41a8010730 tx=0x7f41a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1932838821e93a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41acff9640 1 -- 172.21.15.37:0/2157080135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8004250 con 0x55bd7def9490 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41acff9640 1 -- 172.21.15.37:0/2157080135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a80043f0 con 0x55bd7def9490 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.695+0000 7f41acff9640 1 -- 172.21.15.37:0/2157080135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8003d00 con 0x55bd7def9490 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 -- 172.21.15.37:0/2157080135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 msgr2=0x55bd7defc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 0x55bd7defc750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f41a8010730 tx=0x7f41a8005f90 comp rx=0 tx=0).stop 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 -- 172.21.15.37:0/2157080135 shutdown_connections 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7df04480 0x55bd7df03a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9db0 0x55bd7df00160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 --2- 172.21.15.37:0/2157080135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7def9490 0x55bd7defc750 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 -- 172.21.15.37:0/2157080135 >> 172.21.15.37:0/2157080135 conn(0x55bd7def57b0 msgr2=0x55bd7def49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 -- 172.21.15.37:0/2157080135 shutdown_connections 2024-04-23T18:21:55.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 -- 172.21.15.37:0/2157080135 wait complete. 2024-04-23T18:21:55.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.696+0000 7f41c041e740 1 -- start start 2024-04-23T18:21:55.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 Processor -- start 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7df72990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df72ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df15c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81ae4a0 con 0x55bd7df04480 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81af910 con 0x55bd7def9490 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81aeb70 con 0x55bd7def9db0 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41c041e740 10 client.0 _ll_register_callbacks cb 0x7f41a00a0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df15c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df15c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57068/0 (socket says 172.21.15.37:57068) 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 -- 172.21.15.37:0/156983076 learned_addr learned my addr 172.21.15.37:0/156983076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7df72990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b67fc640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df72ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 -- 172.21.15.37:0/156983076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 msgr2=0x55bd7df72ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df72ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 -- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 msgr2=0x55bd7df72990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7df72990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 -- 172.21.15.37:0/156983076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7df16380 con 0x55bd7df04480 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f41b77fe640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df15c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009ca0 tx=0x7f41b00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201dcf7589c7bcb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.697+0000 7f419e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f419e7fc640 1 -- 172.21.15.37:0/156983076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0031e30 con 0x55bd7df04480 2024-04-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f419e7fc640 1 -- 172.21.15.37:0/156983076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f41b0003b80 con 0x55bd7df04480 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a80043f0 con 0x55bd7df04480 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f419e7fc640 1 -- 172.21.15.37:0/156983076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41b0003d80 con 0x55bd7df04480 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 10 client.15732 fetch_fsmap learned FSMap version 25 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd7df7b210 con 0x55bd7df04480 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f419e7fc640 1 -- 172.21.15.37:0/156983076 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a80043f0 con 0x55bd7df04480 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f419e7fc640 1 -- 172.21.15.37:0/156983076 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f41b002f7f0 con 0x55bd7df04480 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 10 client.15732 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 20 client.15732 populate_metadata read hostname 'smithi037' 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df7b9d0 0x55bd7df7ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 4 client.15732 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1d7c0ffc", "release": true, "await": true}] 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1d7c0ffc", "release": true, "await": true}) -- 0x55bd7df06e80 con 0x55bd7df7b9d0 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.698+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df7b9d0 0x55bd7df7ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.699+0000 7f41b6ffd640 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df7b9d0 0x55bd7df7ddf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33d33cf81392c0 server_cookie=b25b54d0cad03b8e in_seq=0 out_seq=0 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.699+0000 7f419e7fc640 10 client.15732 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.700+0000 7f419e7fc640 1 -- 172.21.15.37:0/156983076 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55bd7df06e80 con 0x55bd7df7b9d0 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.700+0000 7f419e7fc640 10 client.15732 handle_command_reply: tid=0 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.700+0000 7f41c041e740 1 client.15732 shutdown 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.700+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 msgr2=0x55bd7df15c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.700+0000 7f41c041e740 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df15c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009ca0 tx=0x7f41b00095c0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 shutdown_connections 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd7df7b9d0 0x55bd7df7ddf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd7df04480 0x55bd7df15c90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f419e7fc640 0 client.15732 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd7def9db0 0x55bd7df72ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 --2- 172.21.15.37:0/156983076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd7def9490 0x55bd7df72990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 >> 172.21.15.37:0/156983076 conn(0x55bd7def57b0 msgr2=0x55bd7df0f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 shutdown_connections 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 1 -- 172.21.15.37:0/156983076 wait complete. 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41c041e740 20 client.15732 trim_cache size 0 max 16384 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 8, 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "1d7c0ffc": { 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 8, 2024-04-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.3, 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:21:55.701 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:21:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41a40a3e40 msgr2=0x7f41a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41bd708640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41a40a3e40 0x7f41a40a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f41b002f500 tx=0x7f41b002f310 comp rx=0 tx=0).stop 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 msgr2=0x7f41b81a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.701+0000 7f41bd708640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b81a0b60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f41a000d770 tx=0x7f41a000dc30 comp rx=0 tx=0).stop 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 shutdown_connections 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41a40a3e40 0x7f41a40a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b819a730 0x7f41b819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41b8107270 0x7f41b81a0b60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 --2- 172.21.15.37:0/1314042918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41b806a840 0x7f41b81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 >> 172.21.15.37:0/1314042918 conn(0x7f41b8077eb0 msgr2=0x7f41b810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 shutdown_connections 2024-04-23T18:21:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:21:55.702+0000 7f41bd708640 1 -- 172.21.15.37:0/1314042918 wait complete. 2024-04-23T18:21:55.703 INFO:tasks.workunit.client.0.smithi037.stderr:+ dbench 1 2024-04-23T18:21:55.706 INFO:tasks.workunit.client.0.smithi037.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-23T18:21:55.706 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-23T18:21:55.706 INFO:tasks.workunit.client.0.smithi037.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-23T18:21:55.706 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1d7c0ffc', seconds elapsed: 30.4 2024-04-23T18:21:55.707 INFO:tasks.workunit.client.0.smithi037.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-23T18:21:55.807 INFO:tasks.workunit.client.0.smithi037.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-23T18:21:55.807 INFO:tasks.workunit.client.0.smithi037.stdout:releasing clients 2024-04-23T18:21:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:56 smithi172 ceph-mon[32456]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:21:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:56 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3672097264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:56 smithi173 ceph-mon[32906]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:21:56.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:56 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3672097264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:56 smithi037 ceph-mon[29404]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T18:21:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:56 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3672097264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:21:56.806 INFO:tasks.workunit.client.0.smithi037.stdout: 1 3374 35.46 MB/sec warmup 1 sec latency 6.771 ms 2024-04-23T18:21:57.806 INFO:tasks.workunit.client.0.smithi037.stdout: 1 6140 24.04 MB/sec warmup 2 sec latency 16.564 ms 2024-04-23T18:21:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:21:58 smithi172 ceph-mon[32456]: pgmap v48: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 115 KiB/s wr, 22 op/s 2024-04-23T18:21:58.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:21:58 smithi173 ceph-mon[32906]: pgmap v48: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 115 KiB/s wr, 22 op/s 2024-04-23T18:21:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:21:58 smithi037 ceph-mon[29404]: pgmap v48: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 115 KiB/s wr, 22 op/s 2024-04-23T18:21:58.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 8761 21.48 MB/sec warmup 3 sec latency 19.969 ms 2024-04-23T18:21:59.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 11627 19.79 MB/sec warmup 4 sec latency 13.222 ms 2024-04-23T18:22:00.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:00 smithi172 ceph-mon[32456]: pgmap v49: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 115 KiB/s wr, 19 op/s 2024-04-23T18:22:00.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:00 smithi173 ceph-mon[32906]: pgmap v49: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 115 KiB/s wr, 19 op/s 2024-04-23T18:22:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:00 smithi037 ceph-mon[29404]: pgmap v49: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 115 KiB/s wr, 19 op/s 2024-04-23T18:22:00.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 14318 19.22 MB/sec warmup 5 sec latency 20.589 ms 2024-04-23T18:22:01.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 16637 17.43 MB/sec warmup 6 sec latency 57.873 ms 2024-04-23T18:22:02.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:02 smithi172 ceph-mon[32456]: pgmap v50: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 115 KiB/s wr, 19 op/s 2024-04-23T18:22:02.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:02 smithi173 ceph-mon[32906]: pgmap v50: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 115 KiB/s wr, 19 op/s 2024-04-23T18:22:02.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:02 smithi037 ceph-mon[29404]: pgmap v50: 97 pgs: 97 active+clean; 2.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 115 KiB/s wr, 19 op/s 2024-04-23T18:22:02.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 18931 16.94 MB/sec warmup 7 sec latency 13.256 ms 2024-04-23T18:22:03.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 21917 16.97 MB/sec warmup 8 sec latency 7.743 ms 2024-04-23T18:22:04.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:04 smithi172 ceph-mon[32456]: pgmap v51: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 915 KiB/s wr, 134 op/s 2024-04-23T18:22:04.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:04 smithi172 ceph-mon[32456]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T18:22:04.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:04 smithi173 ceph-mon[32906]: pgmap v51: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 915 KiB/s wr, 134 op/s 2024-04-23T18:22:04.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:04 smithi173 ceph-mon[32906]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T18:22:04.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:04 smithi037 ceph-mon[29404]: pgmap v51: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 915 KiB/s wr, 134 op/s 2024-04-23T18:22:04.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:04 smithi037 ceph-mon[29404]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T18:22:04.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 25119 17.17 MB/sec warmup 9 sec latency 17.575 ms 2024-04-23T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.901+0000 7fe9b90bb640 1 -- 172.21.15.37:0/953727218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4101070 msgr2=0x7fe9b4106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.901+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/953727218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4101070 0x7fe9b4106ba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009900 tx=0x7fe99c02a410 comp rx=0 tx=0).stop 2024-04-23T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 -- 172.21.15.37:0/953727218 shutdown_connections 2024-04-23T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/953727218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9b4107280 0x7fe9b410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/953727218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4101070 0x7fe9b4106ba0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/953727218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b41006c0 0x7fe9b4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 -- 172.21.15.37:0/953727218 >> 172.21.15.37:0/953727218 conn(0x7fe9b4077390 msgr2=0x7fe9b40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 -- 172.21.15.37:0/953727218 shutdown_connections 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 -- 172.21.15.37:0/953727218 wait complete. 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 Processor -- start 2024-04-23T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.902+0000 7fe9b90bb640 1 -- start start 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b90bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9b41006c0 0x7fe9b4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b90bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 0x7fe9b4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b90bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b419d000 0x7fe9b419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b90bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41121c0 con 0x7fe9b4107280 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b90bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b4112040 con 0x7fe9b41006c0 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b90bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b4112340 con 0x7fe9b419d000 2024-04-23T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 0x7fe9b4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9b41006c0 0x7fe9b4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 0x7fe9b4198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38486/0 (socket says 172.21.15.37:38486) 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 -- 172.21.15.37:0/664497506 learned_addr learned my addr 172.21.15.37:0/664497506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b359e640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b419d000 0x7fe9b419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 -- 172.21.15.37:0/664497506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b419d000 msgr2=0x7fe9b419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b419d000 0x7fe9b419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 -- 172.21.15.37:0/664497506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9b41006c0 msgr2=0x7fe9b4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9b41006c0 0x7fe9b4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 -- 172.21.15.37:0/664497506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b419daf0 con 0x7fe9b4107280 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9b359e640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b419d000 0x7fe9b419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.903+0000 7fe9abfff640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 0x7fe9b4198c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe99c032fd0 tx=0x7fe99c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.904+0000 7fe9b0d99640 1 -- 172.21.15.37:0/664497506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c03f030 con 0x7fe9b4107280 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.904+0000 7fe9b0d99640 1 -- 172.21.15.37:0/664497506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99c004490 con 0x7fe9b4107280 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.904+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b419dd80 con 0x7fe9b4107280 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.904+0000 7fe9b0d99640 1 -- 172.21.15.37:0/664497506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c035070 con 0x7fe9b4107280 2024-04-23T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.904+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b41a5540 con 0x7fe9b4107280 2024-04-23T18:22:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.906+0000 7fe9b0d99640 1 -- 172.21.15.37:0/664497506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe99c042030 con 0x7fe9b4107280 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.906+0000 7fe9bbdd0740 1 Processor -- start 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 -- start start 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3e924c970 con 0x55b3e92e7f40 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b3e91e94a0 con 0x55b3e92f2800 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b3e9248930 con 0x55b3e92e7620 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38490/0 (socket says 172.21.15.37:38490) 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/1252549646 learned_addr learned my addr 172.21.15.37:0/1252549646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9b359e640 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9abfff640 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.907+0000 7fe9b0d99640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9a00a3dc0 0x7fe9a00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b359e640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9a00a3dc0 0x7fe9a00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/1252549646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 msgr2=0x55b3e92e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/1252549646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 msgr2=0x55b3e92f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/1252549646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e8f5f210 con 0x55b3e92e7f40 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b359e640 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b0d99640 1 -- 172.21.15.37:0/664497506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe99c0f0c60 con 0x7fe9b4107280 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92f22c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400b760 tx=0x7fe9a400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c72c08ff1c03c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9a97fa640 1 -- 172.21.15.37:0/1252549646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4003dd0 con 0x55b3e92e7f40 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9b359e640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9a00a3dc0 0x7fe9a00a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009fd0 tx=0x7fe9ac004ab0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9a97fa640 1 -- 172.21.15.37:0/1252549646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a4015070 con 0x55b3e92e7f40 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.908+0000 7fe9a97fa640 1 -- 172.21.15.37:0/1252549646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4004d90 con 0x55b3e92e7f40 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/1252549646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 msgr2=0x55b3e92f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92f22c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400b760 tx=0x7fe9a400bc20 comp rx=0 tx=0).stop 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/1252549646 shutdown_connections 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92f22c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/1252549646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/1252549646 >> 172.21.15.37:0/1252549646 conn(0x55b3e92e3b30 msgr2=0x55b3e92e3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/1252549646 shutdown_connections 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/1252549646 wait complete. 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.909+0000 7fe9bbdd0740 1 -- start start 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 Processor -- start 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3e92f5430 con 0x55b3e92e7f40 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b3e91e70b0 con 0x55b3e92f2800 2024-04-23T18:22:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b3e92f55b0 con 0x55b3e92e7620 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9bbdd0740 10 client.0 _ll_register_callbacks cb 0x55b3e91d4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:56708/0 (socket says 172.21.15.37:56708) 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/542519442 learned_addr learned my addr 172.21.15.37:0/542519442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9abfff640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b359e640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 msgr2=0x55b3e92ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/542519442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 msgr2=0x55b3e92ef3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92ef3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.910+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/542519442 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e902b620 con 0x55b3e92e7620 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f20f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4003e00 tx=0x7fe9a4003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287cba2f8754cd0f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe99a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe99a7fc640 1 -- 172.21.15.37:0/542519442 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4028060 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe99a7fc640 1 -- 172.21.15.37:0/542519442 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe9a4011850 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a4015070 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe99a7fc640 1 -- 172.21.15.37:0/542519442 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9a40046c0 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 10 client.25300 fetch_fsmap learned FSMap version 25 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3e9301300 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe99a7fc640 1 -- 172.21.15.37:0/542519442 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a4015070 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe99a7fc640 1 -- 172.21.15.37:0/542519442 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe9a4022070 con 0x55b3e92e7620 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 10 client.25300 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 20 client.25300 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e9304af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 4 client.25300 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.911+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3e925a390 con 0x55b3e9302700 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.912+0000 7fe9abfff640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e9304af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.912+0000 7fe9abfff640 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e9304af0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabd085695ec6f6c server_cookie=d771739354f3509e in_seq=0 out_seq=0 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.912+0000 7fe99a7fc640 10 client.25300 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.912+0000 7fe99a7fc640 1 -- 172.21.15.37:0/542519442 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3e925a390 con 0x55b3e9302700 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.912+0000 7fe99a7fc640 10 client.25300 handle_command_reply: tid=0 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 client.25300 shutdown 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 msgr2=0x55b3e92f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f20f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4003e00 tx=0x7fe9a4003e30 comp rx=0 tx=0).stop 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 shutdown_connections 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e9304af0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92f2800 0x55b3e92ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7f40 0x55b3e92ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe99a7fc640 0 client.25300 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/542519442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f20f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 >> 172.21.15.37:0/542519442 conn(0x55b3e92e3b30 msgr2=0x55b3e92e9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 shutdown_connections 2024-04-23T18:22:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/542519442 wait complete. 2024-04-23T18:22:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.913+0000 7fe9bbdd0740 20 client.25300 trim_cache size 0 max 16384 2024-04-23T18:22:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 Processor -- start 2024-04-23T18:22:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 -- start start 2024-04-23T18:22:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 0x55b3e92fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92f2800 0x55b3e930a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41b0490 con 0x55b3e92e7620 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41add60 con 0x55b3e92e7f40 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41abbb0 con 0x55b3e92f2800 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 0x55b3e92fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9b359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92f2800 0x55b3e930a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 0x55b3e92fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38504/0 (socket says 172.21.15.37:38504) 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9b359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92f2800 0x55b3e930a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:56712/0 (socket says 172.21.15.37:56712) 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.929+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/2702112105 learned_addr learned my addr 172.21.15.37:0/2702112105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9abfff640 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/2702112105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92f2800 msgr2=0x55b3e930a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92f2800 0x55b3e930a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/2702112105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 msgr2=0x55b3e92e2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92e2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9b2d9d640 1 -- 172.21.15.37:0/2702112105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e92f4bf0 con 0x55b3e92e7620 2024-04-23T18:22:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 0x55b3e92fe2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4010460 tx=0x7fe9a40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff36a890bf5ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9a8ff9640 1 -- 172.21.15.37:0/2702112105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4028040 con 0x55b3e92e7620 2024-04-23T18:22:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9a8ff9640 1 -- 172.21.15.37:0/2702112105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a4015070 con 0x55b3e92e7620 2024-04-23T18:22:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.930+0000 7fe9a8ff9640 1 -- 172.21.15.37:0/2702112105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4011d60 con 0x55b3e92e7620 2024-04-23T18:22:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2702112105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 msgr2=0x55b3e92fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 0x55b3e92fe2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4010460 tx=0x7fe9a40041a0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2702112105 shutdown_connections 2024-04-23T18:22:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92f2800 0x55b3e930a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92e2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2702112105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92e7620 0x55b3e92fe2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2702112105 >> 172.21.15.37:0/2702112105 conn(0x55b3e92e3b30 msgr2=0x55b3e92ef500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2702112105 shutdown_connections 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2702112105 wait complete. 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 -- start start 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.931+0000 7fe9bbdd0740 1 Processor -- start 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92f2800 0x55b3e9366a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41ac220 con 0x55b3e92f2800 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41ad690 con 0x55b3e92e7f40 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b41ac8f0 con 0x55b3e92e7620 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9bbdd0740 10 client.0 _ll_register_callbacks cb 0x7fe9a4004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92f2800 0x55b3e9366a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:38640/0 (socket says 172.21.15.37:38640) 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9abfff640 1 -- 172.21.15.37:0/2468685432 learned_addr learned my addr 172.21.15.37:0/2468685432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 -- 172.21.15.37:0/2468685432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 msgr2=0x55b3e92f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 -- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 msgr2=0x55b3e92f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 -- 172.21.15.37:0/2468685432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e9366f50 con 0x55b3e92f2800 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9abfff640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe9b359e640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92f2800 0x55b3e9366a10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55b3e92f27d0 tx=0x7fe9ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1660140fa24698a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe99a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.932+0000 7fe99a7fc640 1 -- 172.21.15.37:0/2468685432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac018070 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe99a7fc640 1 -- 172.21.15.37:0/2468685432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe9ac004b20 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a4015070 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe99a7fc640 1 -- 172.21.15.37:0/2468685432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac004d20 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3e9368160 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe99a7fc640 1 -- 172.21.15.37:0/2468685432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a4015070 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe99a7fc640 1 -- 172.21.15.37:0/2468685432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe9ac0055f0 con 0x55b3e92f2800 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 20 client.15750 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e936a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 4 client.15750 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3e9254200 con 0x55b3e9302700 2024-04-23T18:22:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.933+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e936a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.934+0000 7fe9b2d9d640 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e936a900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f292cbfdbcd5bc61 server_cookie=da3301872c8c039 in_seq=0 out_seq=0 2024-04-23T18:22:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.934+0000 7fe99a7fc640 10 client.15750 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe99a7fc640 1 -- 172.21.15.37:0/2468685432 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3e9254200 con 0x55b3e9302700 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe99a7fc640 10 client.15750 handle_command_reply: tid=0 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 client.15750 shutdown 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92f2800 msgr2=0x55b3e9366a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92f2800 0x55b3e9366a10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55b3e92f27d0 tx=0x7fe9ac004a40 comp rx=0 tx=0).stop 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 shutdown_connections 2024-04-23T18:22:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b3e9302700 0x55b3e936a900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3e92f2800 0x55b3e9366a10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3e92e7f40 0x55b3e92f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe99a7fc640 0 client.15750 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 --2- 172.21.15.37:0/2468685432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b3e92e7620 0x55b3e92f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 >> 172.21.15.37:0/2468685432 conn(0x55b3e92e3b30 msgr2=0x55b3e92fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 shutdown_connections 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 1 -- 172.21.15.37:0/2468685432 wait complete. 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9bbdd0740 20 client.15750 trim_cache size 0 max 16384 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:22:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9a00a3dc0 msgr2=0x7fe9a00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9a00a3dc0 0x7fe9a00a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009fd0 tx=0x7fe9ac004ab0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 msgr2=0x7fe9b4198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 0x7fe9b4198c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe99c032fd0 tx=0x7fe99c004270 comp rx=0 tx=0).stop 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.941+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 shutdown_connections 2024-04-23T18:22:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe9a00a3dc0 0x7fe9a00a6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9b419d000 0x7fe9b419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9b4107280 0x7fe9b4198c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 --2- 172.21.15.37:0/664497506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9b41006c0 0x7fe9b4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 >> 172.21.15.37:0/664497506 conn(0x7fe9b4077390 msgr2=0x7fe9b40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 shutdown_connections 2024-04-23T18:22:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:04.942+0000 7fe9b90bb640 1 -- 172.21.15.37:0/664497506 wait complete. 2024-04-23T18:22:05.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 27867 16.74 MB/sec warmup 10 sec latency 15.171 ms 2024-04-23T18:22:06.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:06 smithi172 ceph-mon[32456]: pgmap v52: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 915 KiB/s wr, 131 op/s 2024-04-23T18:22:06.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:06 smithi173 ceph-mon[32906]: pgmap v52: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 915 KiB/s wr, 131 op/s 2024-04-23T18:22:06.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:06 smithi037 ceph-mon[29404]: pgmap v52: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 915 KiB/s wr, 131 op/s 2024-04-23T18:22:06.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 30764 16.45 MB/sec warmup 11 sec latency 48.982 ms 2024-04-23T18:22:07.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 33100 16.26 MB/sec warmup 12 sec latency 52.539 ms 2024-04-23T18:22:08.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:08 smithi172 ceph-mon[32456]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-04-23T18:22:08.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:08 smithi173 ceph-mon[32906]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-04-23T18:22:08.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:08 smithi037 ceph-mon[29404]: pgmap v53: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-04-23T18:22:08.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 36168 16.33 MB/sec warmup 13 sec latency 10.777 ms 2024-04-23T18:22:09.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 39292 16.50 MB/sec warmup 14 sec latency 13.328 ms 2024-04-23T18:22:10.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:10 smithi172 ceph-mon[32456]: pgmap v54: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-23T18:22:10.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:10 smithi173 ceph-mon[32906]: pgmap v54: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-23T18:22:10.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:10 smithi037 ceph-mon[29404]: pgmap v54: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-23T18:22:10.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 41957 16.21 MB/sec warmup 15 sec latency 14.279 ms 2024-04-23T18:22:11.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 44576 16.00 MB/sec warmup 16 sec latency 11.884 ms 2024-04-23T18:22:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:12 smithi172 ceph-mon[32456]: pgmap v55: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-23T18:22:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:12 smithi173 ceph-mon[32906]: pgmap v55: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-23T18:22:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:12 smithi037 ceph-mon[29404]: pgmap v55: 97 pgs: 97 active+clean; 18 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-04-23T18:22:12.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 46955 15.89 MB/sec warmup 17 sec latency 13.482 ms 2024-04-23T18:22:13.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49828 15.96 MB/sec warmup 18 sec latency 57.426 ms 2024-04-23T18:22:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:14 smithi172 ceph-mon[32456]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 284 op/s 2024-04-23T18:22:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:14 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:22:14.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:14 smithi173 ceph-mon[32906]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 284 op/s 2024-04-23T18:22:14.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:14 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:22:14.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:14 smithi037 ceph-mon[29404]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 284 op/s 2024-04-23T18:22:14.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:14 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:22:14.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 49828 15.12 MB/sec warmup 19 sec latency 1057.565 ms 2024-04-23T18:22:15.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50246 14.43 MB/sec warmup 20 sec latency 1506.458 ms 2024-04-23T18:22:16.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:16 smithi172 ceph-mon[32456]: pgmap v57: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-23T18:22:16.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:16 smithi173 ceph-mon[32906]: pgmap v57: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-23T18:22:16.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:16 smithi037 ceph-mon[29404]: pgmap v57: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-04-23T18:22:16.807 INFO:tasks.workunit.client.0.smithi037.stdout: 1 53342 14.59 MB/sec warmup 21 sec latency 26.757 ms 2024-04-23T18:22:17.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 55440 14.25 MB/sec warmup 22 sec latency 17.483 ms 2024-04-23T18:22:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 -- 172.21.15.37:0/3918511594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb78102b90 msgr2=0x7feb78106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/3918511594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb78102b90 0x7feb78106bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009900 tx=0x7feb6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:22:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 -- 172.21.15.37:0/3918511594 shutdown_connections 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/3918511594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 0x7feb7810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/3918511594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb78102b90 0x7feb78106bd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/3918511594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb781021e0 0x7feb781025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 -- 172.21.15.37:0/3918511594 >> 172.21.15.37:0/3918511594 conn(0x7feb780fda00 msgr2=0x7feb780ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 -- 172.21.15.37:0/3918511594 shutdown_connections 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.038+0000 7feb7e8cf640 1 -- 172.21.15.37:0/3918511594 wait complete. 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 Processor -- start 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 -- start start 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb781021e0 0x7feb78198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 0x7feb78198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 0x7feb7819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb781121f0 con 0x7feb7819cfd0 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb78112070 con 0x7feb781072b0 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7e8cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb78112370 con 0x7feb781021e0 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7ce6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 0x7feb7819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7ce6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 0x7feb7819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55754/0 (socket says 172.21.15.37:55754) 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 0x7feb78198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb7ce6c640 1 -- 172.21.15.37:0/2668306987 learned_addr learned my addr 172.21.15.37:0/2668306987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.039+0000 7feb77fff640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb781021e0 0x7feb78198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7ce6c640 1 -- 172.21.15.37:0/2668306987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb781021e0 msgr2=0x7feb78198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7ce6c640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb781021e0 0x7feb78198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7ce6c640 1 -- 172.21.15.37:0/2668306987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 msgr2=0x7feb78198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7ce6c640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 0x7feb78198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7ce6c640 1 -- 172.21.15.37:0/2668306987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7819dac0 con 0x7feb7819cfd0 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb777fe640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 0x7feb78198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7ce6c640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 0x7feb7819d480 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feb7000eb10 tx=0x7feb7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb757fa640 1 -- 172.21.15.37:0/2668306987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70004b20 con 0x7feb7819cfd0 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb757fa640 1 -- 172.21.15.37:0/2668306987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb70004cc0 con 0x7feb7819cfd0 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7819ddb0 con 0x7feb7819cfd0 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb757fa640 1 -- 172.21.15.37:0/2668306987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70010670 con 0x7feb7819cfd0 2024-04-23T18:22:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.040+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781a5570 con 0x7feb7819cfd0 2024-04-23T18:22:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.043+0000 7feb757fa640 1 -- 172.21.15.37:0/2668306987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb70005020 con 0x7feb7819cfd0 2024-04-23T18:22:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.043+0000 7feb815ef740 1 Processor -- start 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.043+0000 7feb815ef740 1 -- start start 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.043+0000 7feb815ef740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f2788a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb815ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb815ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2795bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb815ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5602f26ed970 con 0x5602f2793800 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb815ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5602f268a4a0 con 0x5602f2788f40 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb815ef740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602f26e9930 con 0x5602f2788620 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2795bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27932c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:39294/0 (socket says 172.21.15.37:39294) 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2795bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55762/0 (socket says 172.21.15.37:55762) 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 -- 172.21.15.37:0/1820785330 learned_addr learned my addr 172.21.15.37:0/1820785330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb777fe640 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f2788a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb757fa640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb600a3e40 0x7feb600a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 -- 172.21.15.37:0/1820785330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 msgr2=0x5602f2788a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f2788a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb7ce6c640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb600a3e40 0x7feb600a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 -- 172.21.15.37:0/1820785330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 msgr2=0x5602f2795bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2795bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 -- 172.21.15.37:0/1820785330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f2400210 con 0x5602f2788f40 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb777fe640 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2795bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb757fa640 1 -- 172.21.15.37:0/2668306987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb700d0d10 con 0x7feb7819cfd0 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.044+0000 7feb77fff640 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27932c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb68009d20 tx=0x7feb68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292730c3f349122b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb5e7fc640 1 -- 172.21.15.37:0/1820785330 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6801e070 con 0x5602f2788f40 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb7ce6c640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb600a3e40 0x7feb600a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb700990a0 tx=0x7feb700bd040 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb5e7fc640 1 -- 172.21.15.37:0/1820785330 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb68019ae0 con 0x5602f2788f40 2024-04-23T18:22:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb5e7fc640 1 -- 172.21.15.37:0/1820785330 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800ea50 con 0x5602f2788f40 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 -- 172.21.15.37:0/1820785330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 msgr2=0x5602f27932c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27932c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb68009d20 tx=0x7feb68011810 comp rx=0 tx=0).stop 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 -- 172.21.15.37:0/1820785330 shutdown_connections 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2795bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27932c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 --2- 172.21.15.37:0/1820785330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f2788a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.045+0000 7feb815ef740 1 -- 172.21.15.37:0/1820785330 >> 172.21.15.37:0/1820785330 conn(0x5602f2784b30 msgr2=0x5602f2784f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 -- 172.21.15.37:0/1820785330 shutdown_connections 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 -- 172.21.15.37:0/1820785330 wait complete. 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 -- start start 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 Processor -- start 2024-04-23T18:22:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f27931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2790ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5602f2796430 con 0x5602f2793800 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5602f26880b0 con 0x5602f2788f40 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602f27965b0 con 0x5602f2788620 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb815ef740 10 client.0 _ll_register_callbacks cb 0x5602f2675d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2790ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2790ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55772/0 (socket says 172.21.15.37:55772) 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb777fe640 1 -- 172.21.15.37:0/787238897 learned_addr learned my addr 172.21.15.37:0/787238897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.046+0000 7feb77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27904b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:39306/0 (socket says 172.21.15.37:39306) 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f27931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 -- 172.21.15.37:0/787238897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 msgr2=0x5602f27931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f27931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 -- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 msgr2=0x5602f27904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 -- 172.21.15.37:0/787238897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f24cc620 con 0x5602f2793800 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb777fe640 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2790ac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005ec0 tx=0x7feb6c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8839bf08f1249c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb577fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb577fe640 1 -- 172.21.15.37:0/787238897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c035040 con 0x5602f2793800 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb68019ae0 con 0x5602f2793800 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb577fe640 1 -- 172.21.15.37:0/787238897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb6c005020 con 0x5602f2793800 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb577fe640 1 -- 172.21.15.37:0/787238897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb6c005220 con 0x5602f2793800 2024-04-23T18:22:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb815ef740 10 client.15762 fetch_fsmap learned FSMap version 26 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.047+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602f27a2410 con 0x5602f2793800 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb577fe640 1 -- 172.21.15.37:0/787238897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb68019ae0 con 0x5602f2793800 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb577fe640 1 -- 172.21.15.37:0/787238897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feb6c0035f0 con 0x5602f2793800 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb815ef740 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb815ef740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb815ef740 20 client.15762 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb815ef740 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f27a3880 0x5602f27a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb815ef740 4 client.15762 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602f26fb390 con 0x5602f27a3880 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb77fff640 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f27a3880 0x5602f27a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb77fff640 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f27a3880 0x5602f27a5c70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc2488d1949c3a2 server_cookie=7fd186f92aa47bfa in_seq=0 out_seq=0 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.048+0000 7feb577fe640 10 client.15762 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb577fe640 1 -- 172.21.15.37:0/787238897 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602f26fb390 con 0x5602f27a3880 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb577fe640 10 client.15762 handle_command_reply: tid=0 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 client.15762 shutdown 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 msgr2=0x5602f2790ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2790ac0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005ec0 tx=0x7feb6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 shutdown_connections 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f27a3880 0x5602f27a5c70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2790ac0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788f40 0x5602f27904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 --2- 172.21.15.37:0/787238897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788620 0x5602f27931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 >> 172.21.15.37:0/787238897 conn(0x5602f2784b30 msgr2=0x5602f278a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb577fe640 0 client.15762 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 shutdown_connections 2024-04-23T18:22:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.049+0000 7feb815ef740 1 -- 172.21.15.37:0/787238897 wait complete. 2024-04-23T18:22:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.050+0000 7feb815ef740 20 client.15762 trim_cache size 0 max 16384 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 Processor -- start 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 -- start start 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f278f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f27ab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb781b0560 con 0x5602f2793800 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb781addd0 con 0x5602f2788620 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb815ef740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb781abc20 con 0x5602f2788f40 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f27ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:39318/0 (socket says 172.21.15.37:39318) 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 -- 172.21.15.37:0/1588690205 learned_addr learned my addr 172.21.15.37:0/1588690205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb77fff640 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f278f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 -- 172.21.15.37:0/1588690205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 msgr2=0x5602f278f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f278f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 -- 172.21.15.37:0/1588690205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 msgr2=0x5602f27ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f27ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.067+0000 7feb777fe640 1 -- 172.21.15.37:0/1588690205 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f27802e0 con 0x5602f2788620 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.068+0000 7feb777fe640 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279f2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009900 tx=0x7feb6c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff20181b2bc7599 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.068+0000 7feb5dffb640 1 -- 172.21.15.37:0/1588690205 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c035070 con 0x5602f2788620 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.068+0000 7feb5dffb640 1 -- 172.21.15.37:0/1588690205 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb6c043070 con 0x5602f2788620 2024-04-23T18:22:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.068+0000 7feb5dffb640 1 -- 172.21.15.37:0/1588690205 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c059040 con 0x5602f2788620 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 -- 172.21.15.37:0/1588690205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 msgr2=0x5602f279f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279f2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009900 tx=0x7feb6c000bc0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 -- 172.21.15.37:0/1588690205 shutdown_connections 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f27ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f278f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 --2- 172.21.15.37:0/1588690205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279f2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 -- 172.21.15.37:0/1588690205 >> 172.21.15.37:0/1588690205 conn(0x5602f2784b30 msgr2=0x5602f2783420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 -- 172.21.15.37:0/1588690205 shutdown_connections 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 -- 172.21.15.37:0/1588690205 wait complete. 2024-04-23T18:22:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 -- start start 2024-04-23T18:22:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.069+0000 7feb815ef740 1 Processor -- start 2024-04-23T18:22:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f279ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2798a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb781ac290 con 0x5602f2793800 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb781ad700 con 0x5602f2788620 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb781ac960 con 0x5602f2788f40 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb815ef740 10 client.0 _ll_register_callbacks cb 0x7feb6c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2798a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb77fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f279ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2798a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55790/0 (socket says 172.21.15.37:55790) 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb77fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f279ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:36948/0 (socket says 172.21.15.37:36948) 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 -- 172.21.15.37:0/4043794515 learned_addr learned my addr 172.21.15.37:0/4043794515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 -- 172.21.15.37:0/4043794515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 msgr2=0x5602f279ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f279ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 -- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 msgr2=0x5602f279b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 -- 172.21.15.37:0/4043794515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f27990f0 con 0x5602f2793800 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb77fff640 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f279ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb777fe640 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2798a00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7feb6c006010 tx=0x7feb6c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0adf2d615b363b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.070+0000 7feb577fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb577fe640 1 -- 172.21.15.37:0/4043794515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c059070 con 0x5602f2793800 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb577fe640 1 -- 172.21.15.37:0/4043794515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb6c004970 con 0x5602f2793800 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb6c043070 con 0x5602f2793800 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb577fe640 1 -- 172.21.15.37:0/4043794515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb6c035040 con 0x5602f2793800 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 10 client.15768 fetch_fsmap learned FSMap version 26 2024-04-23T18:22:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602f2809870 con 0x5602f2793800 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb577fe640 1 -- 172.21.15.37:0/4043794515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb6c043070 con 0x5602f2793800 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb577fe640 1 -- 172.21.15.37:0/4043794515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feb6c05f040 con 0x5602f2793800 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 20 client.15768 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f2809f10 0x5602f280c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 4 client.15768 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602f26f5200 con 0x5602f2809f10 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.071+0000 7feb77fff640 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f2809f10 0x5602f280c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.072+0000 7feb77fff640 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f2809f10 0x5602f280c330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd15f50184f422d server_cookie=7a39a51fd42ae0de in_seq=0 out_seq=0 2024-04-23T18:22:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.072+0000 7feb577fe640 10 client.15768 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb577fe640 1 -- 172.21.15.37:0/4043794515 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602f26f5200 con 0x5602f2809f10 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb577fe640 10 client.15768 handle_command_reply: tid=0 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb815ef740 1 client.15768 shutdown 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 msgr2=0x5602f2798a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb815ef740 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2798a00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7feb6c006010 tx=0x7feb6c031400 comp rx=0 tx=0).stop 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 shutdown_connections 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.078+0000 7feb815ef740 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5602f2809f10 0x5602f280c330 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5602f2793800 0x5602f2798a00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602f2788f40 0x5602f279ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 1 --2- 172.21.15.37:0/4043794515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602f2788620 0x5602f279b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb577fe640 0 client.15768 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 >> 172.21.15.37:0/4043794515 conn(0x5602f2784b30 msgr2=0x5602f2783420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 shutdown_connections 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 1 -- 172.21.15.37:0/4043794515 wait complete. 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb815ef740 20 client.15768 trim_cache size 0 max 16384 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:22:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:22:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb600a3e40 msgr2=0x7feb600a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb600a3e40 0x7feb600a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb700990a0 tx=0x7feb700bd040 comp rx=0 tx=0).stop 2024-04-23T18:22:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 msgr2=0x7feb7819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 0x7feb7819d480 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feb7000eb10 tx=0x7feb7000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 shutdown_connections 2024-04-23T18:22:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb600a3e40 0x7feb600a6300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7819cfd0 0x7feb7819d480 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb781072b0 0x7feb78198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 --2- 172.21.15.37:0/2668306987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb781021e0 0x7feb78198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 >> 172.21.15.37:0/2668306987 conn(0x7feb780fda00 msgr2=0x7feb780fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 shutdown_connections 2024-04-23T18:22:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:18.079+0000 7feb7e8cf640 1 -- 172.21.15.37:0/2668306987 wait complete. 2024-04-23T18:22:18.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:18 smithi172 ceph-mon[32456]: pgmap v58: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-23T18:22:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:18 smithi173 ceph-mon[32906]: pgmap v58: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-23T18:22:18.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:18 smithi037 ceph-mon[29404]: pgmap v58: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-23T18:22:18.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57459 14.22 MB/sec warmup 23 sec latency 17.363 ms 2024-04-23T18:22:19.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 60559 14.41 MB/sec warmup 24 sec latency 16.077 ms 2024-04-23T18:22:20.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:20 smithi172 ceph-mon[32456]: pgmap v59: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-04-23T18:22:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:20 smithi173 ceph-mon[32906]: pgmap v59: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-04-23T18:22:20.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:20 smithi037 ceph-mon[29404]: pgmap v59: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-04-23T18:22:20.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 62772 14.13 MB/sec warmup 25 sec latency 18.581 ms 2024-04-23T18:22:21.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 64756 14.07 MB/sec warmup 26 sec latency 12.870 ms 2024-04-23T18:22:22.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:22 smithi172 ceph-mon[32456]: pgmap v60: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-04-23T18:22:22.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:22 smithi173 ceph-mon[32906]: pgmap v60: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-04-23T18:22:22.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:22 smithi037 ceph-mon[29404]: pgmap v60: 97 pgs: 97 active+clean; 29 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-04-23T18:22:22.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 67759 14.23 MB/sec warmup 27 sec latency 9.711 ms 2024-04-23T18:22:23.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 70442 14.17 MB/sec warmup 28 sec latency 16.826 ms 2024-04-23T18:22:24.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:24 smithi172 ceph-mon[32456]: pgmap v61: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-04-23T18:22:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:24 smithi173 ceph-mon[32906]: pgmap v61: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-04-23T18:22:24.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:24 smithi037 ceph-mon[29404]: pgmap v61: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-04-23T18:22:24.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 72183 13.98 MB/sec warmup 29 sec latency 25.131 ms 2024-04-23T18:22:25.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 74950 14.09 MB/sec warmup 30 sec latency 11.200 ms 2024-04-23T18:22:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:26 smithi172 ceph-mon[32456]: pgmap v62: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-04-23T18:22:26.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:26 smithi173 ceph-mon[32906]: pgmap v62: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-04-23T18:22:26.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:26 smithi037 ceph-mon[29404]: pgmap v62: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-04-23T18:22:26.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 77838 14.10 MB/sec warmup 31 sec latency 13.026 ms 2024-04-23T18:22:27.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 79273 13.87 MB/sec warmup 32 sec latency 39.014 ms 2024-04-23T18:22:28.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:28 smithi172 ceph-mon[32456]: pgmap v63: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.8 MiB/s wr, 262 op/s 2024-04-23T18:22:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:28 smithi173 ceph-mon[32906]: pgmap v63: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.8 MiB/s wr, 262 op/s 2024-04-23T18:22:28.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:28 smithi037 ceph-mon[29404]: pgmap v63: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.8 MiB/s wr, 262 op/s 2024-04-23T18:22:28.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 81952 13.96 MB/sec warmup 33 sec latency 16.245 ms 2024-04-23T18:22:29.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 84078 13.76 MB/sec warmup 34 sec latency 14.805 ms 2024-04-23T18:22:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:30 smithi172 ceph-mon[32456]: pgmap v64: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-04-23T18:22:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:30 smithi173 ceph-mon[32906]: pgmap v64: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-04-23T18:22:30.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:30 smithi037 ceph-mon[29404]: pgmap v64: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-04-23T18:22:30.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 85796 13.71 MB/sec warmup 35 sec latency 13.020 ms 2024-04-23T18:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 -- 172.21.15.37:0/342164653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 msgr2=0x7fd34c10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 --2- 172.21.15.37:0/342164653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c10ee70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd344009ca0 tx=0x7fd34402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 -- 172.21.15.37:0/342164653 shutdown_connections 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 --2- 172.21.15.37:0/342164653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c10ee70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 --2- 172.21.15.37:0/342164653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd34c102b80 0x7fd34c106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 --2- 172.21.15.37:0/342164653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c1021d0 0x7fd34c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 -- 172.21.15.37:0/342164653 >> 172.21.15.37:0/342164653 conn(0x7fd34c0fda50 msgr2=0x7fd34c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 -- 172.21.15.37:0/342164653 shutdown_connections 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.169+0000 7fd35282a640 1 -- 172.21.15.37:0/342164653 wait complete. 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 Processor -- start 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 -- start start 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd34c1021d0 0x7fd34c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c19cf00 0x7fd34c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c112210 con 0x7fd34c1072a0 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c112090 con 0x7fd34c1021d0 2024-04-23T18:22:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd35282a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c112390 con 0x7fd34c19cf00 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd34b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd34b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35416/0 (socket says 172.21.15.37:35416) 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd34b7fe640 1 -- 172.21.15.37:0/2878024977 learned_addr learned my addr 172.21.15.37:0/2878024977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.170+0000 7fd34bfff640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd34c1021d0 0x7fd34c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd350dc7640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c19cf00 0x7fd34c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd34b7fe640 1 -- 172.21.15.37:0/2878024977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c19cf00 msgr2=0x7fd34c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c19cf00 0x7fd34c19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd34b7fe640 1 -- 172.21.15.37:0/2878024977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd34c1021d0 msgr2=0x7fd34c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd34c1021d0 0x7fd34c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd34b7fe640 1 -- 172.21.15.37:0/2878024977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c19d9f0 con 0x7fd34c1072a0 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c198bd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e9c0 tx=0x7fd34000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd3497fa640 1 -- 172.21.15.37:0/2878024977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34000cd80 con 0x7fd34c1072a0 2024-04-23T18:22:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd3497fa640 1 -- 172.21.15.37:0/2878024977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3400054d0 con 0x7fd34c1072a0 2024-04-23T18:22:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c19dce0 con 0x7fd34c1072a0 2024-04-23T18:22:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd3497fa640 1 -- 172.21.15.37:0/2878024977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340010430 con 0x7fd34c1072a0 2024-04-23T18:22:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.171+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1a54d0 con 0x7fd34c1072a0 2024-04-23T18:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.173+0000 7fd3497fa640 1 -- 172.21.15.37:0/2878024977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd340004fd0 con 0x7fd34c1072a0 2024-04-23T18:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 Processor -- start 2024-04-23T18:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 -- start start 2024-04-23T18:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd3497fa640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3340a3dc0 0x7fd3340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 0x5555c56c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0f40 0x5555c56cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555c5625970 con 0x5555c56c0620 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555c55c24a0 con 0x5555c56cb800 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd35561b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5555c5621930 con 0x5555c56c0f40 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd350dc7640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3340a3dc0 0x7fd3340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd34bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 0x5555c56c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd34bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 0x5555c56c0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35418/0 (socket says 172.21.15.37:35418) 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd34bfff640 1 -- 172.21.15.37:0/3235664894 learned_addr learned my addr 172.21.15.37:0/3235664894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd350dc7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.174+0000 7fd3497fa640 1 -- 172.21.15.37:0/2878024977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fd3400d0c90 con 0x7fd34c1072a0 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0f40 0x5555c56cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34bfff640 1 -- 172.21.15.37:0/3235664894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0f40 msgr2=0x5555c56cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34bfff640 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0f40 0x5555c56cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34bfff640 1 -- 172.21.15.37:0/3235664894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 msgr2=0x5555c56cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34bfff640 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34bfff640 1 -- 172.21.15.37:0/3235664894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c5338210 con 0x5555c56c0620 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd350dc7640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3340a3dc0 0x7fd3340a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd34402f500 tx=0x7fd34402f3d0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd34bfff640 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 0x5555c56c0a00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b6e0 tx=0x7fd33c00bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ee660f5325e7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd3327fc640 1 -- 172.21.15.37:0/3235664894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003b50 con 0x5555c56c0620 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd3327fc640 1 -- 172.21.15.37:0/3235664894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd33c015070 con 0x5555c56c0620 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.175+0000 7fd3327fc640 1 -- 172.21.15.37:0/3235664894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c004c60 con 0x5555c56c0620 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 -- 172.21.15.37:0/3235664894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 msgr2=0x5555c56c0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 0x5555c56c0a00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b6e0 tx=0x7fd33c00bba0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 -- 172.21.15.37:0/3235664894 shutdown_connections 2024-04-23T18:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0f40 0x5555c56cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 --2- 172.21.15.37:0/3235664894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0620 0x5555c56c0a00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 -- 172.21.15.37:0/3235664894 >> 172.21.15.37:0/3235664894 conn(0x5555c56bcb30 msgr2=0x5555c56bcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 -- 172.21.15.37:0/3235664894 shutdown_connections 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 -- 172.21.15.37:0/3235664894 wait complete. 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 -- start start 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.176+0000 7fd35561b740 1 Processor -- start 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0620 0x5555c56cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56cb800 0x5555c56c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555c56ce430 con 0x5555c56c0f40 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555c55c00b0 con 0x5555c56c0620 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5555c56ce5b0 con 0x5555c56cb800 2024-04-23T18:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd35561b740 10 client.0 _ll_register_callbacks cb 0x5555c55add30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35420/0 (socket says 172.21.15.37:35420) 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0620 0x5555c56cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 -- 172.21.15.37:0/1948797920 learned_addr learned my addr 172.21.15.37:0/1948797920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd350dc7640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56cb800 0x5555c56c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 -- 172.21.15.37:0/1948797920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56cb800 msgr2=0x5555c56c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56cb800 0x5555c56c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 -- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0620 msgr2=0x5555c56cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0620 0x5555c56cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 -- 172.21.15.37:0/1948797920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c5404620 con 0x5555c56c0f40 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34bfff640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0620 0x5555c56cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56c83f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd34009cb50 tx=0x7fd34009c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ea37297bf0815a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd32b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.177+0000 7fd32b7fe640 1 -- 172.21.15.37:0/1948797920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34009d2c0 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd32b7fe640 1 -- 172.21.15.37:0/1948797920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fd3400ce5f0 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33c015070 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd32b7fe640 1 -- 172.21.15.37:0/1948797920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd3400ce7f0 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5555c56da300 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd32b7fe640 1 -- 172.21.15.37:0/1948797920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd33c015070 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd32b7fe640 1 -- 172.21.15.37:0/1948797920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd3400a53a0 con 0x5555c56c0f40 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 20 client.15780 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c56db880 0x5555c56ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 4 client.15780 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:22:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c5633390 con 0x5555c56db880 2024-04-23T18:22:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.178+0000 7fd34bfff640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c56db880 0x5555c56ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.179+0000 7fd32b7fe640 10 client.15780 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.179+0000 7fd34bfff640 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c56db880 0x5555c56ddcd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9b1f1291bfdcfa server_cookie=add4478a5ffb4d08 in_seq=0 out_seq=0 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd32b7fe640 1 -- 172.21.15.37:0/1948797920 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555c5633390 con 0x5555c56db880 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd32b7fe640 10 client.15780 handle_command_reply: tid=0 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 client.15780 shutdown 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 msgr2=0x5555c56c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56c83f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd34009cb50 tx=0x7fd34009c650 comp rx=0 tx=0).stop 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 shutdown_connections 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c56db880 0x5555c56ddcd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56cb800 0x5555c56c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56c83f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd32b7fe640 0 client.15780 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 --2- 172.21.15.37:0/1948797920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0620 0x5555c56cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 >> 172.21.15.37:0/1948797920 conn(0x5555c56bcb30 msgr2=0x5555c56c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 shutdown_connections 2024-04-23T18:22:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.182+0000 7fd35561b740 1 -- 172.21.15.37:0/1948797920 wait complete. 2024-04-23T18:22:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.183+0000 7fd35561b740 20 client.15780 trim_cache size 0 max 16384 2024-04-23T18:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.204+0000 7fd35561b740 1 Processor -- start 2024-04-23T18:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.204+0000 7fd35561b740 1 -- start start 2024-04-23T18:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.204+0000 7fd35561b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.204+0000 7fd35561b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0f40 0x5555c56c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd35561b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 0x5555c56cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd35561b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b04e0 con 0x5555c56cb800 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd35561b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1add50 con 0x5555c56c0f40 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd35561b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1abba0 con 0x5555c56c0620 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 0x5555c56cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 0x5555c56cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35424/0 (socket says 172.21.15.37:35424) 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 -- 172.21.15.37:0/3604674507 learned_addr learned my addr 172.21.15.37:0/3604674507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0f40 0x5555c56c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd34bfff640 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 -- 172.21.15.37:0/3604674507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 msgr2=0x5555c56d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 -- 172.21.15.37:0/3604674507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0f40 msgr2=0x5555c56c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0f40 0x5555c56c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 -- 172.21.15.37:0/3604674507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c56d7e20 con 0x5555c56cb800 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd350dc7640 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 0x5555c56cac30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd34402ade0 tx=0x7fd344005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b2b64f9af625f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd331ffb640 1 -- 172.21.15.37:0/3604674507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344031590 con 0x5555c56cb800 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd331ffb640 1 -- 172.21.15.37:0/3604674507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd344050070 con 0x5555c56cb800 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.205+0000 7fd331ffb640 1 -- 172.21.15.37:0/3604674507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344004670 con 0x5555c56cb800 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 -- 172.21.15.37:0/3604674507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 msgr2=0x5555c56cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 0x5555c56cac30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd34402ade0 tx=0x7fd344005c30 comp rx=0 tx=0).stop 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 -- 172.21.15.37:0/3604674507 shutdown_connections 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56cb800 0x5555c56cac30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56c0f40 0x5555c56c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 --2- 172.21.15.37:0/3604674507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 -- 172.21.15.37:0/3604674507 >> 172.21.15.37:0/3604674507 conn(0x5555c56bcb30 msgr2=0x5555c56c3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 -- 172.21.15.37:0/3604674507 shutdown_connections 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 -- 172.21.15.37:0/3604674507 wait complete. 2024-04-23T18:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.206+0000 7fd35561b740 1 -- start start 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 Processor -- start 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1ac210 con 0x5555c56c0f40 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1ad680 con 0x5555c56cb800 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1ac8e0 con 0x5555c56c0620 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd35561b740 10 client.0 _ll_register_callbacks cb 0x7fd34009c790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35438/0 (socket says 172.21.15.37:35438) 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34b7fe640 1 -- 172.21.15.37:0/871280766 learned_addr learned my addr 172.21.15.37:0/871280766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd350dc7640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34bfff640 1 -- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 msgr2=0x5555c56ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34bfff640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34bfff640 1 -- 172.21.15.37:0/871280766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 msgr2=0x5555c56d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34bfff640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34bfff640 1 -- 172.21.15.37:0/871280766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c56cf4d0 con 0x5555c56c0620 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.207+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd34bfff640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd33c004880 tx=0x7fd33c00b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26073b83318be72c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd32b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd32b7fe640 1 -- 172.21.15.37:0/871280766 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c02c070 con 0x5555c56c0620 2024-04-23T18:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd32b7fe640 1 -- 172.21.15.37:0/871280766 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fd33c0280e0 con 0x5555c56c0620 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd344050070 con 0x5555c56c0620 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd32b7fe640 1 -- 172.21.15.37:0/871280766 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd33c038050 con 0x5555c56c0620 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd35561b740 10 client.25342 fetch_fsmap learned FSMap version 26 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5555c5741a20 con 0x5555c56c0620 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.208+0000 7fd32b7fe640 1 -- 172.21.15.37:0/871280766 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd344050070 con 0x5555c56c0620 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd32b7fe640 1 -- 172.21.15.37:0/871280766 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd33c0317e0 con 0x5555c56c0620 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd35561b740 10 client.25342 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd35561b740 10 client.25342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd35561b740 20 client.25342 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd35561b740 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c5741ec0 0x5555c57442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd35561b740 4 client.25342 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555c562d200 con 0x5555c5741ec0 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c5741ec0 0x5555c57442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd34b7fe640 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c5741ec0 0x5555c57442b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71486d584ee5a187 server_cookie=533a8bae0467c644 in_seq=0 out_seq=0 2024-04-23T18:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.209+0000 7fd32b7fe640 10 client.25342 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd32b7fe640 1 -- 172.21.15.37:0/871280766 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555c562d200 con 0x5555c5741ec0 2024-04-23T18:22:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd32b7fe640 10 client.25342 handle_command_reply: tid=0 2024-04-23T18:22:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 client.25342 shutdown 2024-04-23T18:22:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 msgr2=0x5555c56d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd33c004880 tx=0x7fd33c00b420 comp rx=0 tx=0).stop 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 shutdown_connections 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555c5741ec0 0x5555c57442b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555c56cb800 0x5555c56ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555c56c0f40 0x5555c56d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd32b7fe640 0 client.25342 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 --2- 172.21.15.37:0/871280766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555c56c0620 0x5555c56d3770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 >> 172.21.15.37:0/871280766 conn(0x5555c56bcb30 msgr2=0x5555c56bbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 shutdown_connections 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 1 -- 172.21.15.37:0/871280766 wait complete. 2024-04-23T18:22:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.241+0000 7fd35561b740 20 client.25342 trim_cache size 0 max 16384 2024-04-23T18:22:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:22:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:22:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:22:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3340a3dc0 msgr2=0x7fd3340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3340a3dc0 0x7fd3340a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd34402f500 tx=0x7fd34402f3d0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 msgr2=0x7fd34c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c198bd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e9c0 tx=0x7fd34000ee80 comp rx=0 tx=0).stop 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 shutdown_connections 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd3340a3dc0 0x7fd3340a6280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c19cf00 0x7fd34c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd34c1072a0 0x7fd34c198bd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 --2- 172.21.15.37:0/2878024977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd34c1021d0 0x7fd34c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 >> 172.21.15.37:0/2878024977 conn(0x7fd34c0fda50 msgr2=0x7fd34c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 shutdown_connections 2024-04-23T18:22:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:31.242+0000 7fd35282a640 1 -- 172.21.15.37:0/2878024977 wait complete. 2024-04-23T18:22:31.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 88687 13.82 MB/sec warmup 36 sec latency 16.286 ms 2024-04-23T18:22:32.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:32 smithi173 ceph-mon[32906]: pgmap v65: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-04-23T18:22:32.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:32 smithi037 ceph-mon[29404]: pgmap v65: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-04-23T18:22:32.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:32 smithi172 ceph-mon[32456]: pgmap v65: 97 pgs: 97 active+clean; 42 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-04-23T18:22:32.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89662 13.51 MB/sec warmup 37 sec latency 76.636 ms 2024-04-23T18:22:33.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 92512 13.63 MB/sec warmup 38 sec latency 12.213 ms 2024-04-23T18:22:34.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:34 smithi173 ceph-mon[32906]: pgmap v66: 97 pgs: 97 active+clean; 66 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-04-23T18:22:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:34 smithi037 ceph-mon[29404]: pgmap v66: 97 pgs: 97 active+clean; 66 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-04-23T18:22:34.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:34 smithi172 ceph-mon[32456]: pgmap v66: 97 pgs: 97 active+clean; 66 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-04-23T18:22:34.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 94410 13.43 MB/sec warmup 39 sec latency 11.361 ms 2024-04-23T18:22:35.808 INFO:tasks.workunit.client.0.smithi037.stdout: 1 96369 13.43 MB/sec warmup 40 sec latency 14.249 ms 2024-04-23T18:22:36.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:36 smithi173 ceph-mon[32906]: pgmap v67: 97 pgs: 97 active+clean; 66 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 236 op/s 2024-04-23T18:22:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:36 smithi037 ceph-mon[29404]: pgmap v67: 97 pgs: 97 active+clean; 66 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 236 op/s 2024-04-23T18:22:36.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:36 smithi172 ceph-mon[32456]: pgmap v67: 97 pgs: 97 active+clean; 66 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 236 op/s 2024-04-23T18:22:36.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 99165 13.46 MB/sec warmup 41 sec latency 31.124 ms 2024-04-23T18:22:37.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:37 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:37 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:37 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 100275 13.27 MB/sec warmup 42 sec latency 81.945 ms 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: pgmap v68: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 370 op/s 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:38.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:38 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: pgmap v68: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 370 op/s 2024-04-23T18:22:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:38.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:38.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:38 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: pgmap v68: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 370 op/s 2024-04-23T18:22:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:38.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:38.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:38 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:22:38.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 103116 13.38 MB/sec warmup 43 sec latency 23.554 ms 2024-04-23T18:22:39.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 104605 13.20 MB/sec warmup 44 sec latency 13.346 ms 2024-04-23T18:22:40.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:40 smithi173 ceph-mon[32906]: pgmap v69: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-04-23T18:22:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:40 smithi037 ceph-mon[29404]: pgmap v69: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-04-23T18:22:40.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:40 smithi172 ceph-mon[32456]: pgmap v69: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-04-23T18:22:40.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 106978 13.22 MB/sec warmup 45 sec latency 9.330 ms 2024-04-23T18:22:41.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 109822 13.24 MB/sec warmup 46 sec latency 12.320 ms 2024-04-23T18:22:42.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:42 smithi173 ceph-mon[32906]: pgmap v70: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-04-23T18:22:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:42 smithi037 ceph-mon[29404]: pgmap v70: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-04-23T18:22:42.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:42 smithi172 ceph-mon[32456]: pgmap v70: 97 pgs: 97 active+clean; 70 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-04-23T18:22:42.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 110593 13.06 MB/sec warmup 47 sec latency 206.443 ms 2024-04-23T18:22:43.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 113494 13.15 MB/sec warmup 48 sec latency 32.569 ms 2024-04-23T18:22:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 -- 172.21.15.37:0/879638288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52341010f0 msgr2=0x7f52341014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 --2- 172.21.15.37:0/879638288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52341010f0 0x7f52341014d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5224009900 tx=0x7f522402a410 comp rx=0 tx=0).stop 2024-04-23T18:22:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 -- 172.21.15.37:0/879638288 shutdown_connections 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 --2- 172.21.15.37:0/879638288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5234106170 0x7f523410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 --2- 172.21.15.37:0/879638288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5234101aa0 0x7f5234105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 --2- 172.21.15.37:0/879638288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52341010f0 0x7f52341014d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 -- 172.21.15.37:0/879638288 >> 172.21.15.37:0/879638288 conn(0x7f5234077fd0 msgr2=0x7f52340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 -- 172.21.15.37:0/879638288 shutdown_connections 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.330+0000 7f5239e10640 1 -- 172.21.15.37:0/879638288 wait complete. 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 Processor -- start 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 -- start start 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52341010f0 0x7f52341987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5234106170 0x7f5234198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 0x7f523419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52341110b0 con 0x7f523419d010 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5234110f30 con 0x7f5234106170 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5239e10640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5234111230 con 0x7f52341010f0 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5233fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 0x7f523419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5233fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 0x7f523419d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39332/0 (socket says 172.21.15.37:39332) 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5233fff640 1 -- 172.21.15.37:0/3763489725 learned_addr learned my addr 172.21.15.37:0/3763489725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f5232ffd640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5234106170 0x7f5234198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.331+0000 7f52337fe640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52341010f0 0x7f52341987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5233fff640 1 -- 172.21.15.37:0/3763489725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52341010f0 msgr2=0x7f52341987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5233fff640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52341010f0 0x7f52341987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5233fff640 1 -- 172.21.15.37:0/3763489725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5234106170 msgr2=0x7f5234198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5233fff640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5234106170 0x7f5234198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5233fff640 1 -- 172.21.15.37:0/3763489725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523419db00 con 0x7f523419d010 2024-04-23T18:22:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5233fff640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 0x7f523419d4c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f522c00eb10 tx=0x7f522c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5230ff9640 1 -- 172.21.15.37:0/3763489725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c004b20 con 0x7f523419d010 2024-04-23T18:22:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523419ddf0 con 0x7f523419d010 2024-04-23T18:22:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5230ff9640 1 -- 172.21.15.37:0/3763489725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c004cc0 con 0x7f523419d010 2024-04-23T18:22:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5230ff9640 1 -- 172.21.15.37:0/3763489725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c010670 con 0x7f523419d010 2024-04-23T18:22:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.332+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341a55e0 con 0x7f523419d010 2024-04-23T18:22:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.334+0000 7f5230ff9640 1 -- 172.21.15.37:0/3763489725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f522c005020 con 0x7f523419d010 2024-04-23T18:22:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.334+0000 7f523cc02740 1 Processor -- start 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 -- start start 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5230ff9640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f521c0a3dc0 0x7f521c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb103d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb104c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb104f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcb0fa2970 con 0x55bcb103de20 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bcb0f3f4a0 con 0x55bcb104cc10 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f523cc02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bcb0f9e930 con 0x55bcb103d500 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5232ffd640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f521c0a3dc0 0x7f521c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f52337fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb103d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb104f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5230ff9640 1 -- 172.21.15.37:0/3763489725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f522c0d0d10 con 0x7f523419d010 2024-04-23T18:22:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f52337fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb103d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:47282/0 (socket says 172.21.15.37:47282) 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f52337fe640 1 -- 172.21.15.37:0/2959991951 learned_addr learned my addr 172.21.15.37:0/2959991951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5232ffd640 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb104c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 -- 172.21.15.37:0/2959991951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 msgr2=0x55bcb103d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb103d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 -- 172.21.15.37:0/2959991951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 msgr2=0x55bcb104c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb104c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 -- 172.21.15.37:0/2959991951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb0cb5210 con 0x55bcb104cc10 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f52337fe640 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb103d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5232ffd640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f521c0a3dc0 0x7f521c0a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f523406a6e0 tx=0x7f5228009b60 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.335+0000 7f5233fff640 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb104f000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f522c0ce450 tx=0x7f522c0988e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3724f8abdc298f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f5219ffb640 1 -- 172.21.15.37:0/2959991951 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0bbe60 con 0x55bcb104cc10 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f5219ffb640 1 -- 172.21.15.37:0/2959991951 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c09d3a0 con 0x55bcb104cc10 2024-04-23T18:22:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f5219ffb640 1 -- 172.21.15.37:0/2959991951 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0ce6d0 con 0x55bcb104cc10 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 -- 172.21.15.37:0/2959991951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 msgr2=0x55bcb104f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb104f000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f522c0ce450 tx=0x7f522c0988e0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 -- 172.21.15.37:0/2959991951 shutdown_connections 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb104f000 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb104c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 --2- 172.21.15.37:0/2959991951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb103d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 -- 172.21.15.37:0/2959991951 >> 172.21.15.37:0/2959991951 conn(0x55bcb1039b30 msgr2=0x55bcb1039f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 -- 172.21.15.37:0/2959991951 shutdown_connections 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.336+0000 7f523cc02740 1 -- 172.21.15.37:0/2959991951 wait complete. 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 -- start start 2024-04-23T18:22:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 Processor -- start 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb104c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1049770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 0x55bcb1049cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcb104f840 con 0x55bcb104cc10 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bcb0f3d0b0 con 0x55bcb103de20 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bcb104f9c0 con 0x55bcb103d500 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f523cc02740 10 client.0 _ll_register_callbacks cb 0x55bcb0f2ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f5233fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 0x55bcb1049cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f52337fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb104c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f5232ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1049770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f5233fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 0x55bcb1049cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39356/0 (socket says 172.21.15.37:39356) 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f5233fff640 1 -- 172.21.15.37:0/3308891660 learned_addr learned my addr 172.21.15.37:0/3308891660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.337+0000 7f52337fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb104c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:47294/0 (socket says 172.21.15.37:47294) 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5233fff640 1 -- 172.21.15.37:0/3308891660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 msgr2=0x55bcb104c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5233fff640 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb104c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5233fff640 1 -- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 msgr2=0x55bcb1049770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5233fff640 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1049770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5233fff640 1 -- 172.21.15.37:0/3308891660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb0d81620 con 0x55bcb104cc10 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5233fff640 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 0x55bcb1049cb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f522c0ce500 tx=0x7f522c0a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83bf7feb1409b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5212ffd640 1 -- 172.21.15.37:0/3308891660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0bbe60 con 0x55bcb104cc10 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5212ffd640 1 -- 172.21.15.37:0/3308891660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f522c0bd0e0 con 0x55bcb104cc10 2024-04-23T18:22:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c09d3a0 con 0x55bcb104cc10 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5212ffd640 1 -- 172.21.15.37:0/3308891660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f522c0bd320 con 0x55bcb104cc10 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 10 client.15798 fetch_fsmap learned FSMap version 26 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcb105b6f0 con 0x55bcb104cc10 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5212ffd640 1 -- 172.21.15.37:0/3308891660 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c09d3a0 con 0x55bcb104cc10 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f5212ffd640 1 -- 172.21.15.37:0/3308891660 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f522c09cf00 con 0x55bcb104cc10 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 20 client.15798 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.338+0000 7f523cc02740 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb105ca40 0x55bcb105ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.339+0000 7f523cc02740 4 client.15798 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.339+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcb0fb0390 con 0x55bcb105ca40 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.339+0000 7f52337fe640 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb105ca40 0x55bcb105ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.339+0000 7f52337fe640 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb105ca40 0x55bcb105ee30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18329516e12b1a server_cookie=fc2dff046eced99c in_seq=0 out_seq=0 2024-04-23T18:22:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.339+0000 7f5212ffd640 10 client.15798 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f5212ffd640 1 -- 172.21.15.37:0/3308891660 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcb0fb0390 con 0x55bcb105ca40 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f5212ffd640 10 client.15798 handle_command_reply: tid=0 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 client.15798 shutdown 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 msgr2=0x55bcb1049cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 0x55bcb1049cb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f522c0ce500 tx=0x7f522c0a5910 comp rx=0 tx=0).stop 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 shutdown_connections 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb105ca40 0x55bcb105ee30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb104cc10 0x55bcb1049cb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1049770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 --2- 172.21.15.37:0/3308891660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb104c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 >> 172.21.15.37:0/3308891660 conn(0x55bcb1039b30 msgr2=0x55bcb103f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f5212ffd640 0 client.15798 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 shutdown_connections 2024-04-23T18:22:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 1 -- 172.21.15.37:0/3308891660 wait complete. 2024-04-23T18:22:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.340+0000 7f523cc02740 20 client.15798 trim_cache size 0 max 16384 2024-04-23T18:22:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.355+0000 7f523cc02740 1 Processor -- start 2024-04-23T18:22:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.355+0000 7f523cc02740 1 -- start start 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f523cc02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb1058740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f523cc02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb1044f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f523cc02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb1038b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f523cc02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52341ae320 con 0x55bcb103de20 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f523cc02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52341abb90 con 0x55bcb104cc10 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f523cc02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52341a99e0 con 0x55bcb103d500 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb1044f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb1044f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39368/0 (socket says 172.21.15.37:39368) 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 -- 172.21.15.37:0/2139033318 learned_addr learned my addr 172.21.15.37:0/2139033318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f52337fe640 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb1058740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5233fff640 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb1038b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 -- 172.21.15.37:0/2139033318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 msgr2=0x55bcb1058740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb1058740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 -- 172.21.15.37:0/2139033318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 msgr2=0x55bcb1038b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb1038b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 -- 172.21.15.37:0/2139033318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb1039960 con 0x55bcb103de20 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5233fff640 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb1038b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f5232ffd640 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb1044f00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5228009eb0 tx=0x7f5228021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721300ebb2f883d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.356+0000 7f52197fa640 1 -- 172.21.15.37:0/2139033318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522802e070 con 0x55bcb103de20 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f52197fa640 1 -- 172.21.15.37:0/2139033318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52280035f0 con 0x55bcb103de20 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f52197fa640 1 -- 172.21.15.37:0/2139033318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280044e0 con 0x55bcb103de20 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 -- 172.21.15.37:0/2139033318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 msgr2=0x55bcb1044f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb1044f00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5228009eb0 tx=0x7f5228021000 comp rx=0 tx=0).stop 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 -- 172.21.15.37:0/2139033318 shutdown_connections 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb104cc10 0x55bcb1038b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103de20 0x55bcb1044f00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 --2- 172.21.15.37:0/2139033318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb103d500 0x55bcb1058740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 -- 172.21.15.37:0/2139033318 >> 172.21.15.37:0/2139033318 conn(0x55bcb1039b30 msgr2=0x55bcb1038420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 -- 172.21.15.37:0/2139033318 shutdown_connections 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.357+0000 7f523cc02740 1 -- 172.21.15.37:0/2139033318 wait complete. 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 -- start start 2024-04-23T18:22:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 Processor -- start 2024-04-23T18:22:44.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:44 smithi173 ceph-mon[32906]: pgmap v71: 97 pgs: 97 active+clean; 74 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 406 op/s 2024-04-23T18:22:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 0x55bcb1054c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1055190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb104cc10 0x55bcb1045ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52341aa050 con 0x55bcb103d500 2024-04-23T18:22:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52341ab4c0 con 0x55bcb103de20 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52341aa720 con 0x55bcb104cc10 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f523cc02740 10 client.0 _ll_register_callbacks cb 0x7f5228005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 0x55bcb1054c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 0x55bcb1054c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39374/0 (socket says 172.21.15.37:39374) 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 -- 172.21.15.37:0/1883875189 learned_addr learned my addr 172.21.15.37:0/1883875189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f5232ffd640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1055190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f5233fff640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb104cc10 0x55bcb1045ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 -- 172.21.15.37:0/1883875189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb104cc10 msgr2=0x55bcb1045ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb104cc10 0x55bcb1045ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 -- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 msgr2=0x55bcb1055190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1055190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.358+0000 7f52337fe640 1 -- 172.21.15.37:0/1883875189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb1046370 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f52337fe640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 0x55bcb1054c50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f522402a930 tx=0x7f5224032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1085a0db2a7031 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f5212ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f5212ffd640 1 -- 172.21.15.37:0/1883875189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224035040 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f5212ffd640 1 -- 172.21.15.37:0/1883875189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f5224004240 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52280035f0 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f5212ffd640 1 -- 172.21.15.37:0/1883875189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5224055050 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcb10c27e0 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f5212ffd640 1 -- 172.21.15.37:0/1883875189 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52280035f0 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f5212ffd640 1 -- 172.21.15.37:0/1883875189 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52240464b0 con 0x55bcb103d500 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 20 client.15804 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb10c2f70 0x55bcb10c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 4 client.15804 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:22:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.359+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcb0faa200 con 0x55bcb10c2f70 2024-04-23T18:22:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.360+0000 7f5232ffd640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb10c2f70 0x55bcb10c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.360+0000 7f5212ffd640 10 client.15804 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.360+0000 7f5232ffd640 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb10c2f70 0x55bcb10c5390 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b431a9fd805aa1cd server_cookie=390a450d3e4e9c36 in_seq=0 out_seq=0 2024-04-23T18:22:44.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:44 smithi037 ceph-mon[29404]: pgmap v71: 97 pgs: 97 active+clean; 74 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 406 op/s 2024-04-23T18:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f5212ffd640 1 -- 172.21.15.37:0/1883875189 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcb0faa200 con 0x55bcb10c2f70 2024-04-23T18:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f5212ffd640 10 client.15804 handle_command_reply: tid=0 2024-04-23T18:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 client.15804 shutdown 2024-04-23T18:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 msgr2=0x55bcb1054c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 0x55bcb1054c50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f522402a930 tx=0x7f5224032c40 comp rx=0 tx=0).stop 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 shutdown_connections 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcb10c2f70 0x55bcb10c5390 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcb104cc10 0x55bcb1045ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcb103de20 0x55bcb1055190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f5212ffd640 0 client.15804 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 --2- 172.21.15.37:0/1883875189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcb103d500 0x55bcb1054c50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 >> 172.21.15.37:0/1883875189 conn(0x55bcb1039b30 msgr2=0x55bcb1038cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 shutdown_connections 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 1 -- 172.21.15.37:0/1883875189 wait complete. 2024-04-23T18:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.392+0000 7f523cc02740 20 client.15804 trim_cache size 0 max 16384 2024-04-23T18:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f521c0a3dc0 msgr2=0x7f521c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f521c0a3dc0 0x7f521c0a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f523406a6e0 tx=0x7f5228009b60 comp rx=0 tx=0).stop 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 msgr2=0x7f523419d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 0x7f523419d4c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f522c00eb10 tx=0x7f522c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 shutdown_connections 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f521c0a3dc0 0x7f521c0a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523419d010 0x7f523419d4c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5234106170 0x7f5234198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 --2- 172.21.15.37:0/3763489725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52341010f0 0x7f52341987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 >> 172.21.15.37:0/3763489725 conn(0x7f5234077fd0 msgr2=0x7f5234104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 shutdown_connections 2024-04-23T18:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:44.393+0000 7f5239e10640 1 -- 172.21.15.37:0/3763489725 wait complete. 2024-04-23T18:22:44.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:44 smithi172 ceph-mon[32456]: pgmap v71: 97 pgs: 97 active+clean; 74 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.6 MiB/s wr, 406 op/s 2024-04-23T18:22:44.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114969 13.02 MB/sec warmup 49 sec latency 16.025 ms 2024-04-23T18:22:45.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 117470 13.04 MB/sec warmup 50 sec latency 10.689 ms 2024-04-23T18:22:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:46 smithi037 ceph-mon[29404]: pgmap v72: 97 pgs: 97 active+clean; 74 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-04-23T18:22:46.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:46 smithi172 ceph-mon[32456]: pgmap v72: 97 pgs: 97 active+clean; 74 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-04-23T18:22:46.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:46 smithi173 ceph-mon[32906]: pgmap v72: 97 pgs: 97 active+clean; 74 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-04-23T18:22:46.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 119869 13.00 MB/sec warmup 51 sec latency 15.415 ms 2024-04-23T18:22:47.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 121517 12.92 MB/sec warmup 52 sec latency 30.977 ms 2024-04-23T18:22:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:48 smithi037 ceph-mon[29404]: pgmap v73: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-04-23T18:22:48.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:48 smithi172 ceph-mon[32456]: pgmap v73: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-04-23T18:22:48.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:48 smithi173 ceph-mon[32906]: pgmap v73: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-04-23T18:22:48.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 124388 13.02 MB/sec warmup 53 sec latency 23.328 ms 2024-04-23T18:22:49.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 125872 12.88 MB/sec warmup 54 sec latency 15.453 ms 2024-04-23T18:22:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:50 smithi037 ceph-mon[29404]: pgmap v74: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-04-23T18:22:50.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:50 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:22:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:50 smithi172 ceph-mon[32456]: pgmap v74: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-04-23T18:22:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:50 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:22:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:50 smithi173 ceph-mon[32906]: pgmap v74: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-04-23T18:22:50.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:50 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:22:50.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 128185 12.90 MB/sec warmup 55 sec latency 10.113 ms 2024-04-23T18:22:51.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 130695 12.89 MB/sec warmup 56 sec latency 33.496 ms 2024-04-23T18:22:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:52 smithi037 ceph-mon[29404]: pgmap v75: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-04-23T18:22:52.406 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:52 smithi173 ceph-mon[32906]: pgmap v75: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-04-23T18:22:52.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:52 smithi172 ceph-mon[32456]: pgmap v75: 97 pgs: 97 active+clean; 78 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 296 op/s 2024-04-23T18:22:52.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 132021 12.79 MB/sec warmup 57 sec latency 82.091 ms 2024-04-23T18:22:53.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 134832 12.86 MB/sec warmup 58 sec latency 23.075 ms 2024-04-23T18:22:54.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:54 smithi037 ceph-mon[29404]: pgmap v76: 97 pgs: 97 active+clean; 83 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 418 op/s 2024-04-23T18:22:54.388 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:54 smithi172 ceph-mon[32456]: pgmap v76: 97 pgs: 97 active+clean; 83 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 418 op/s 2024-04-23T18:22:54.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:54 smithi173 ceph-mon[32906]: pgmap v76: 97 pgs: 97 active+clean; 83 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 418 op/s 2024-04-23T18:22:54.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 136288 12.76 MB/sec warmup 59 sec latency 17.540 ms 2024-04-23T18:22:55.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 138753 12.78 MB/sec warmup 60 sec latency 11.853 ms 2024-04-23T18:22:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:56 smithi037 ceph-mon[29404]: pgmap v77: 97 pgs: 97 active+clean; 83 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-04-23T18:22:56.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:56 smithi172 ceph-mon[32456]: pgmap v77: 97 pgs: 97 active+clean; 83 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-04-23T18:22:56.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:56 smithi173 ceph-mon[32906]: pgmap v77: 97 pgs: 97 active+clean; 83 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 279 op/s 2024-04-23T18:22:56.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 140919 12.70 MB/sec warmup 61 sec latency 14.500 ms 2024-04-23T18:22:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 -- 172.21.15.37:0/4029599708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 msgr2=0x7f8a6010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 --2- 172.21.15.37:0/4029599708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a6010dd40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009be0 tx=0x7f8a5802a6c0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 -- 172.21.15.37:0/4029599708 shutdown_connections 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 --2- 172.21.15.37:0/4029599708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a6010dd40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 --2- 172.21.15.37:0/4029599708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a60101aa0 0x7f8a60105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 --2- 172.21.15.37:0/4029599708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601010f0 0x7f8a601014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 -- 172.21.15.37:0/4029599708 >> 172.21.15.37:0/4029599708 conn(0x7f8a60077fd0 msgr2=0x7f8a600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 -- 172.21.15.37:0/4029599708 shutdown_connections 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.486+0000 7f8a65376640 1 -- 172.21.15.37:0/4029599708 wait complete. 2024-04-23T18:22:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 Processor -- start 2024-04-23T18:22:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 -- start start 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a601010f0 0x7f8a60199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a60199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a6019a010 0x7f8a601a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601110c0 con 0x7f8a60106170 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60110f40 con 0x7f8a601010f0 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a65376640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8a60111240 con 0x7f8a6019a010 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a60199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a601010f0 0x7f8a60199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a5f7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a6019a010 0x7f8a601a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a601010f0 0x7f8a60199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:51044/0 (socket says 172.21.15.37:51044) 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.487+0000 7f8a5effd640 1 -- 172.21.15.37:0/743613421 learned_addr learned my addr 172.21.15.37:0/743613421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5f7fe640 1 -- 172.21.15.37:0/743613421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a601010f0 msgr2=0x7f8a60199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a601010f0 0x7f8a60199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5f7fe640 1 -- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 msgr2=0x7f8a60199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a60199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5f7fe640 1 -- 172.21.15.37:0/743613421 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a601a4d90 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a60199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a6019a010 0x7f8a601a45c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a5802abe0 tx=0x7f8a58003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a57fff640 1 -- 172.21.15.37:0/743613421 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58041020 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a57fff640 1 -- 172.21.15.37:0/743613421 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a58003e10 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a57fff640 1 -- 172.21.15.37:0/743613421 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58036040 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a601a5020 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.488+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a601a5490 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a57fff640 1 -- 172.21.15.37:0/743613421 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a58044020 con 0x7f8a6019a010 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 Processor -- start 2024-04-23T18:22:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 -- start start 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a57fff640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8a4c0a3e40 0x7f8a4c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6b8800 0x55ca5f6babf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca5f612970 con 0x55ca5f6b8800 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca5f5af4a0 con 0x55ca5f6ad620 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.491+0000 7f8a6816f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ca5f60e930 con 0x55ca5f6adf40 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:51046/0 (socket says 172.21.15.37:51046) 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6b8800 0x55ca5f6babf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:38458/0 (socket says 172.21.15.37:38458) 2024-04-23T18:22:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/1707344078 learned_addr learned my addr 172.21.15.37:0/1707344078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8a4c0a3e40 0x7f8a4c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a57fff640 1 -- 172.21.15.37:0/743613421 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8a580f1230 con 0x7f8a6019a010 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 -- 172.21.15.37:0/1707344078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 msgr2=0x55ca5f6ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 -- 172.21.15.37:0/1707344078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6b8800 msgr2=0x55ca5f6babf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6b8800 0x55ca5f6babf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 -- 172.21.15.37:0/1707344078 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5f325210 con 0x55ca5f6adf40 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5effd640 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b82c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009c80 tx=0x7f8a50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf1725d35545bad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8a4c0a3e40 0x7f8a4c0a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009fd0 tx=0x7f8a48009260 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a557fa640 1 -- 172.21.15.37:0/1707344078 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50004230 con 0x55ca5f6adf40 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a557fa640 1 -- 172.21.15.37:0/1707344078 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a500043d0 con 0x55ca5f6adf40 2024-04-23T18:22:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.492+0000 7f8a557fa640 1 -- 172.21.15.37:0/1707344078 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50004cd0 con 0x55ca5f6adf40 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 -- 172.21.15.37:0/1707344078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 msgr2=0x55ca5f6b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b82c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009c80 tx=0x7f8a50011810 comp rx=0 tx=0).stop 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 -- 172.21.15.37:0/1707344078 shutdown_connections 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6b8800 0x55ca5f6babf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b82c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1707344078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 -- 172.21.15.37:0/1707344078 >> 172.21.15.37:0/1707344078 conn(0x55ca5f6a9b30 msgr2=0x55ca5f6a9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 -- 172.21.15.37:0/1707344078 shutdown_connections 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 -- 172.21.15.37:0/1707344078 wait complete. 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.493+0000 7f8a6816f740 1 -- start start 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 Processor -- start 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca5f6bb430 con 0x55ca5f6adf40 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca5f5ad0b0 con 0x55ca5f6ad620 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ca5f6bb5b0 con 0x55ca5f6b8800 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a6816f740 10 client.0 _ll_register_callbacks cb 0x55ca5f59ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:51048/0 (socket says 172.21.15.37:51048) 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46206/0 (socket says 172.21.15.37:46206) 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 -- 172.21.15.37:0/3529950521 learned_addr learned my addr 172.21.15.37:0/3529950521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 -- 172.21.15.37:0/3529950521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 msgr2=0x55ca5f6b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 -- 172.21.15.37:0/3529950521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 msgr2=0x55ca5f6b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5effd640 1 -- 172.21.15.37:0/3529950521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5f3f1620 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.494+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a5effd640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6b8130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a50004770 tx=0x7f8a500198c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f0f1a1f94d6e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a467fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a467fc640 1 -- 172.21.15.37:0/3529950521 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50028020 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a467fc640 1 -- 172.21.15.37:0/3529950521 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8a50024420 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a500043d0 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a467fc640 1 -- 172.21.15.37:0/3529950521 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a50033050 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 10 client.25391 fetch_fsmap learned FSMap version 27 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca5f6c7340 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a467fc640 1 -- 172.21.15.37:0/3529950521 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a500043d0 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a467fc640 1 -- 172.21.15.37:0/3529950521 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f8a50018070 con 0x55ca5f6ad620 2024-04-23T18:22:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 10 client.25391 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 20 client.25391 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f6c8740 0x55ca5f6cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 4 client.25391 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.495+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca5f620390 con 0x55ca5f6c8740 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.496+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f6c8740 0x55ca5f6cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.496+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f6c8740 0x55ca5f6cab30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11bff32f5aedee17 server_cookie=f89154c338665c1e in_seq=0 out_seq=0 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.496+0000 7f8a467fc640 10 client.25391 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a467fc640 1 -- 172.21.15.37:0/3529950521 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca5f620390 con 0x55ca5f6c8740 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a467fc640 10 client.25391 handle_command_reply: tid=0 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 client.25391 shutdown 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 msgr2=0x55ca5f6b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6b8130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a50004770 tx=0x7f8a500198c0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 shutdown_connections 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f6c8740 0x55ca5f6cab30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a467fc640 0 client.25391 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 --2- 172.21.15.37:0/3529950521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6b8130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 >> 172.21.15.37:0/3529950521 conn(0x55ca5f6a9b30 msgr2=0x55ca5f6af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 shutdown_connections 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 1 -- 172.21.15.37:0/3529950521 wait complete. 2024-04-23T18:22:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.497+0000 7f8a6816f740 20 client.25391 trim_cache size 0 max 16384 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 Processor -- start 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 -- start start 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6ad620 0x55ca5f6c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6b8800 0x55ca5f6a8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601b0450 con 0x55ca5f6adf40 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601adcc0 con 0x55ca5f6b8800 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a6816f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601abb10 con 0x55ca5f6ad620 2024-04-23T18:22:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6ad620 0x55ca5f6c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46208/0 (socket says 172.21.15.37:46208) 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/1423878840 learned_addr learned my addr 172.21.15.37:0/1423878840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6b8800 0x55ca5f6a8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/1423878840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6ad620 msgr2=0x55ca5f6c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6ad620 0x55ca5f6c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/1423878840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6b8800 msgr2=0x55ca5f6a8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6b8800 0x55ca5f6a8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.514+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/1423878840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5f6b5cc0 con 0x55ca5f6adf40 2024-04-23T18:22:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.515+0000 7f8a5effd640 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6ad620 0x55ca5f6c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:22:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.515+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6b8800 0x55ca5f6a8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:22:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.515+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b0af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005fd0 tx=0x7f8a480095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fcb3cd830f7c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.515+0000 7f8a54ff9640 1 -- 172.21.15.37:0/1423878840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48004020 con 0x55ca5f6adf40 2024-04-23T18:22:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.515+0000 7f8a54ff9640 1 -- 172.21.15.37:0/1423878840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4802a8e0 con 0x55ca5f6adf40 2024-04-23T18:22:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.515+0000 7f8a54ff9640 1 -- 172.21.15.37:0/1423878840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480052a0 con 0x55ca5f6adf40 2024-04-23T18:22:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- 172.21.15.37:0/1423878840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 msgr2=0x55ca5f6b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b0af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8a48005fd0 tx=0x7f8a480095b0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- 172.21.15.37:0/1423878840 shutdown_connections 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6b8800 0x55ca5f6a8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6b0af0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- 172.21.15.37:0/1423878840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6ad620 0x55ca5f6c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- 172.21.15.37:0/1423878840 >> 172.21.15.37:0/1423878840 conn(0x55ca5f6a9b30 msgr2=0x55ca5f6a8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- 172.21.15.37:0/1423878840 shutdown_connections 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- 172.21.15.37:0/1423878840 wait complete. 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- start start 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 Processor -- start 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ac180 con 0x55ca5f6adf40 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.516+0000 7f8a6816f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ad5f0 con 0x55ca5f6ad620 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a6816f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601ac850 con 0x55ca5f6b8800 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a6816f740 10 client.0 _ll_register_callbacks cb 0x7f8a50009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6bbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46220/0 (socket says 172.21.15.37:46220) 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/2906042781 learned_addr learned my addr 172.21.15.37:0/2906042781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5effd640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5f7fe640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/2906042781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 msgr2=0x55ca5f6bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6bc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 msgr2=0x55ca5f6bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 -- 172.21.15.37:0/2906042781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5f72cb70 con 0x55ca5f6adf40 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a5e7fc640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6bbd00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8a4802d040 tx=0x7f8a4800cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3524360e6421c580 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a467fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:22:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a467fc640 1 -- 172.21.15.37:0/2906042781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4802f070 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a467fc640 1 -- 172.21.15.37:0/2906042781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8a48028050 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4802a8e0 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.517+0000 7f8a467fc640 1 -- 172.21.15.37:0/2906042781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8a4803f050 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 10 client.15822 fetch_fsmap learned FSMap version 27 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca5f72dd50 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a467fc640 1 -- 172.21.15.37:0/2906042781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4802a8e0 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a467fc640 1 -- 172.21.15.37:0/2906042781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8350+0+0 (secure 0 0 0) 0x7f8a4801f040 con 0x55ca5f6adf40 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 20 client.15822 populate_metadata read hostname 'smithi037' 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f72e4c0 0x55ca5f7308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 4 client.15822 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:22:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca5f61a200 con 0x55ca5f72e4c0 2024-04-23T18:22:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.518+0000 7f8a5effd640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f72e4c0 0x55ca5f7308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.519+0000 7f8a5effd640 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f72e4c0 0x55ca5f7308e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e239a95ce66c81c server_cookie=4f7e1c44830a10 in_seq=0 out_seq=0 2024-04-23T18:22:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.519+0000 7f8a467fc640 10 client.15822 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a467fc640 1 -- 172.21.15.37:0/2906042781 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca5f61a200 con 0x55ca5f72e4c0 2024-04-23T18:22:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a467fc640 10 client.15822 handle_command_reply: tid=0 2024-04-23T18:22:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 client.15822 shutdown 2024-04-23T18:22:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 msgr2=0x55ca5f6bbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6bbd00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8a4802d040 tx=0x7f8a4800cfd0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 shutdown_connections 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ca5f72e4c0 0x55ca5f7308e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca5f6b8800 0x55ca5f6bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca5f6adf40 0x55ca5f6bbd00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a467fc640 0 client.15822 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 --2- 172.21.15.37:0/2906042781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca5f6ad620 0x55ca5f6bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 >> 172.21.15.37:0/2906042781 conn(0x55ca5f6a9b30 msgr2=0x55ca5f6b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 shutdown_connections 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 1 -- 172.21.15.37:0/2906042781 wait complete. 2024-04-23T18:22:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.559+0000 7f8a6816f740 20 client.15822 trim_cache size 0 max 16384 2024-04-23T18:22:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:22:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:22:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:22:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:22:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8a4c0a3e40 msgr2=0x7f8a4c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8a4c0a3e40 0x7f8a4c0a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009fd0 tx=0x7f8a48009260 comp rx=0 tx=0).stop 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a6019a010 msgr2=0x7f8a601a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a6019a010 0x7f8a601a45c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a5802abe0 tx=0x7f8a58003c20 comp rx=0 tx=0).stop 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 shutdown_connections 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8a4c0a3e40 0x7f8a4c0a6300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a6019a010 0x7f8a601a45c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a60106170 0x7f8a60199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 --2- 172.21.15.37:0/743613421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a601010f0 0x7f8a60199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 >> 172.21.15.37:0/743613421 conn(0x7f8a60077fd0 msgr2=0x7f8a600fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 shutdown_connections 2024-04-23T18:22:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:22:57.560+0000 7f8a65376640 1 -- 172.21.15.37:0/743613421 wait complete. 2024-04-23T18:22:57.809 INFO:tasks.workunit.client.0.smithi037.stdout: 1 142517 12.68 MB/sec warmup 62 sec latency 25.504 ms 2024-04-23T18:22:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:22:58 smithi037 ceph-mon[29404]: pgmap v78: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 410 op/s 2024-04-23T18:22:58.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:22:58 smithi172 ceph-mon[32456]: pgmap v78: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 410 op/s 2024-04-23T18:22:58.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:22:58 smithi173 ceph-mon[32906]: pgmap v78: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 410 op/s 2024-04-23T18:22:58.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 145437 12.75 MB/sec warmup 63 sec latency 20.236 ms 2024-04-23T18:22:59.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 146915 12.66 MB/sec warmup 64 sec latency 11.967 ms 2024-04-23T18:23:00.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:00 smithi172 ceph-mon[32456]: pgmap v79: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-04-23T18:23:00.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:00 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:23:00.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:00 smithi173 ceph-mon[32906]: pgmap v79: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-04-23T18:23:00.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:00 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:23:00.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:00 smithi037 ceph-mon[29404]: pgmap v79: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-04-23T18:23:00.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:00 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:23:00.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 149153 12.66 MB/sec warmup 65 sec latency 20.870 ms 2024-04-23T18:23:01.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 150453 12.56 MB/sec warmup 66 sec latency 16.842 ms 2024-04-23T18:23:02.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:02 smithi037 ceph-mon[29404]: pgmap v80: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-04-23T18:23:02.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:02 smithi172 ceph-mon[32456]: pgmap v80: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-04-23T18:23:02.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:02 smithi173 ceph-mon[32906]: pgmap v80: 97 pgs: 97 active+clean; 87 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-04-23T18:23:02.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 152876 12.58 MB/sec warmup 67 sec latency 30.821 ms 2024-04-23T18:23:03.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 154810 12.49 MB/sec warmup 68 sec latency 77.423 ms 2024-04-23T18:23:04.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:04 smithi172 ceph-mon[32456]: pgmap v81: 97 pgs: 97 active+clean; 92 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-04-23T18:23:04.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:04 smithi173 ceph-mon[32906]: pgmap v81: 97 pgs: 97 active+clean; 92 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-04-23T18:23:04.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:04 smithi037 ceph-mon[29404]: pgmap v81: 97 pgs: 97 active+clean; 92 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-04-23T18:23:04.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 156666 12.50 MB/sec warmup 69 sec latency 22.613 ms 2024-04-23T18:23:05.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 159477 12.53 MB/sec warmup 70 sec latency 13.866 ms 2024-04-23T18:23:06.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:06 smithi172 ceph-mon[32456]: pgmap v82: 97 pgs: 97 active+clean; 92 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 313 op/s 2024-04-23T18:23:06.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:06 smithi173 ceph-mon[32906]: pgmap v82: 97 pgs: 97 active+clean; 92 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 313 op/s 2024-04-23T18:23:06.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:06 smithi037 ceph-mon[29404]: pgmap v82: 97 pgs: 97 active+clean; 92 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 313 op/s 2024-04-23T18:23:06.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 160861 12.45 MB/sec warmup 71 sec latency 14.060 ms 2024-04-23T18:23:07.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 163479 12.50 MB/sec warmup 72 sec latency 14.512 ms 2024-04-23T18:23:08.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:08 smithi172 ceph-mon[32456]: pgmap v83: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 438 op/s 2024-04-23T18:23:08.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:08 smithi173 ceph-mon[32906]: pgmap v83: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 438 op/s 2024-04-23T18:23:08.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:08 smithi037 ceph-mon[29404]: pgmap v83: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 438 op/s 2024-04-23T18:23:08.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 164994 12.41 MB/sec warmup 73 sec latency 41.588 ms 2024-04-23T18:23:09.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 167189 12.43 MB/sec warmup 74 sec latency 10.387 ms 2024-04-23T18:23:10.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:10 smithi172 ceph-mon[32456]: pgmap v84: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-04-23T18:23:10.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:10 smithi173 ceph-mon[32906]: pgmap v84: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-04-23T18:23:10.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:10 smithi037 ceph-mon[29404]: pgmap v84: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-04-23T18:23:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 -- 172.21.15.37:0/3957301300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0101aa0 msgr2=0x7f92b0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 --2- 172.21.15.37:0/3957301300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0101aa0 0x7f92b0105a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009900 tx=0x7f92a002a410 comp rx=0 tx=0).stop 2024-04-23T18:23:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 -- 172.21.15.37:0/3957301300 shutdown_connections 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 --2- 172.21.15.37:0/3957301300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b0106170 0x7f92b010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 --2- 172.21.15.37:0/3957301300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0101aa0 0x7f92b0105a90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 --2- 172.21.15.37:0/3957301300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b01010f0 0x7f92b01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 -- 172.21.15.37:0/3957301300 >> 172.21.15.37:0/3957301300 conn(0x7f92b0077fd0 msgr2=0x7f92b00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 -- 172.21.15.37:0/3957301300 shutdown_connections 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.656+0000 7f92b7472640 1 -- 172.21.15.37:0/3957301300 wait complete. 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 Processor -- start 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 -- start start 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b01010f0 0x7f92b01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 0x7f92b0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b019cf90 0x7f92b019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0112220 con 0x7f92b0106170 2024-04-23T18:23:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01120a0 con 0x7f92b019cf90 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b7472640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01123a0 con 0x7f92b01010f0 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b4a0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 0x7f92b0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b4a0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 0x7f92b0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42490/0 (socket says 172.21.15.37:42490) 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b4a0d640 1 -- 172.21.15.37:0/3692895964 learned_addr learned my addr 172.21.15.37:0/3692895964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b520e640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b01010f0 0x7f92b01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.657+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b019cf90 0x7f92b019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b4a0d640 1 -- 172.21.15.37:0/3692895964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b01010f0 msgr2=0x7f92b01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b01010f0 0x7f92b01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b4a0d640 1 -- 172.21.15.37:0/3692895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b019cf90 msgr2=0x7f92b019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b019cf90 0x7f92b019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b4a0d640 1 -- 172.21.15.37:0/3692895964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b019da80 con 0x7f92b0106170 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b520e640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b01010f0 0x7f92b01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b019cf90 0x7f92b019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 0x7f92b0198c10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f92a0032fd0 tx=0x7f92a0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92a67fc640 1 -- 172.21.15.37:0/3692895964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a003f030 con 0x7f92b0106170 2024-04-23T18:23:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b019dd10 con 0x7f92b0106170 2024-04-23T18:23:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92a67fc640 1 -- 172.21.15.37:0/3692895964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a0004490 con 0x7f92b0106170 2024-04-23T18:23:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92a67fc640 1 -- 172.21.15.37:0/3692895964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0035070 con 0x7f92b0106170 2024-04-23T18:23:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.658+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01a54d0 con 0x7f92b0106170 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.660+0000 7f92a67fc640 1 -- 172.21.15.37:0/3692895964 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92a0042030 con 0x7f92b0106170 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 Processor -- start 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 -- start start 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d65bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9cbd970 con 0x55d8c9d63800 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9c5a4a0 con 0x55d8c9d58f40 2024-04-23T18:23:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92ba186740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9cb9930 con 0x55d8c9d58620 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92a67fc640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f929c0a3dc0 0x7f929c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92b5a0f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92b5a0f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d58a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:41258/0 (socket says 172.21.15.37:41258) 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92b5a0f640 1 -- 172.21.15.37:0/2367865821 learned_addr learned my addr 172.21.15.37:0/2367865821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.661+0000 7f92b520e640 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d65bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f929c0a3dc0 0x7f929c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92a67fc640 1 -- 172.21.15.37:0/3692895964 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f92a00f0b30 con 0x7f92b0106170 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 -- 172.21.15.37:0/2367865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 msgr2=0x55d8c9d632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 -- 172.21.15.37:0/2367865821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 msgr2=0x55d8c9d65bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d65bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 -- 172.21.15.37:0/2367865821 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c99d0210 con 0x55d8c9d58620 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d58a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009fd0 tx=0x7f92a800f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb3a24535f7f1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f929c0a3dc0 0x7f929c0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92a801e000 tx=0x7f92a8020000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f929b7fe640 1 -- 172.21.15.37:0/2367865821 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8013860 con 0x55d8c9d58620 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f929b7fe640 1 -- 172.21.15.37:0/2367865821 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a8023020 con 0x55d8c9d58620 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.662+0000 7f929b7fe640 1 -- 172.21.15.37:0/2367865821 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a801fd20 con 0x55d8c9d58620 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 -- 172.21.15.37:0/2367865821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 msgr2=0x55d8c9d58a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d58a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009fd0 tx=0x7f92a800f7d0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 -- 172.21.15.37:0/2367865821 shutdown_connections 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d65bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 --2- 172.21.15.37:0/2367865821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d58a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 -- 172.21.15.37:0/2367865821 >> 172.21.15.37:0/2367865821 conn(0x55d8c9d54b30 msgr2=0x55d8c9d54f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 -- 172.21.15.37:0/2367865821 shutdown_connections 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.663+0000 7f92ba186740 1 -- 172.21.15.37:0/2367865821 wait complete. 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 -- start start 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 Processor -- start 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 0x55d8c9d631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d63800 0x55d8c9d60ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9d6a7e0 con 0x55d8c9d58620 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9c580b0 con 0x55d8c9d58f40 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9d6a960 con 0x55d8c9d63800 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92ba186740 10 client.0 _ll_register_callbacks cb 0x55d8c9c45d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 0x55d8c9d631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b5a0f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d63800 0x55d8c9d60ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 0x55d8c9d631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42500/0 (socket says 172.21.15.37:42500) 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 -- 172.21.15.37:0/648700318 learned_addr learned my addr 172.21.15.37:0/648700318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 -- 172.21.15.37:0/648700318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d63800 msgr2=0x55d8c9d60ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d63800 0x55d8c9d60ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 -- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 msgr2=0x55d8c9d604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b520e640 1 -- 172.21.15.37:0/648700318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c9a9c620 con 0x55d8c9d58620 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.664+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d63800 0x55d8c9d60ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92b520e640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 0x55d8c9d631c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f92ac012d20 tx=0x7f92ac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7605f3a92c6410 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f9298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f9298ff9640 1 -- 172.21.15.37:0/648700318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac012110 con 0x55d8c9d58620 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f9298ff9640 1 -- 172.21.15.37:0/648700318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f92ac004050 con 0x55d8c9d58620 2024-04-23T18:23:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a8023020 con 0x55d8c9d58620 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f9298ff9640 1 -- 172.21.15.37:0/648700318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92ac0122b0 con 0x55d8c9d58620 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8c9d76820 con 0x55d8c9d58620 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f9298ff9640 1 -- 172.21.15.37:0/648700318 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a8023020 con 0x55d8c9d58620 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f9298ff9640 1 -- 172.21.15.37:0/648700318 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f92ac010430 con 0x55d8c9d58620 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 20 client.15834 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9d77ce0 0x55d8c9d7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 4 client.15834 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.665+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8c9ccb390 con 0x55d8c9d77ce0 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.666+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9d77ce0 0x55d8c9d7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.666+0000 7f9298ff9640 10 client.15834 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.666+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9d77ce0 0x55d8c9d7a0d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5e68f1230066a9 server_cookie=2b89ccf419b8547 in_seq=0 out_seq=0 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f9298ff9640 1 -- 172.21.15.37:0/648700318 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8c9ccb390 con 0x55d8c9d77ce0 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f9298ff9640 10 client.15834 handle_command_reply: tid=0 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 client.15834 shutdown 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 msgr2=0x55d8c9d631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 0x55d8c9d631c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f92ac012d20 tx=0x7f92ac00f790 comp rx=0 tx=0).stop 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 shutdown_connections 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9d77ce0 0x55d8c9d7a0d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d63800 0x55d8c9d60ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f9298ff9640 0 client.15834 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 --2- 172.21.15.37:0/648700318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d58620 0x55d8c9d631c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 >> 172.21.15.37:0/648700318 conn(0x55d8c9d54b30 msgr2=0x55d8c9d5a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 shutdown_connections 2024-04-23T18:23:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 1 -- 172.21.15.37:0/648700318 wait complete. 2024-04-23T18:23:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.667+0000 7f92ba186740 20 client.15834 trim_cache size 0 max 16384 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 Processor -- start 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 -- start start 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d7f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01ae210 con 0x55d8c9d63800 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01aba80 con 0x55d8c9d58f40 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92ba186740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01a98d0 con 0x55d8c9d58620 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b5a0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d7f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b520e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b5a0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d7f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42514/0 (socket says 172.21.15.37:42514) 2024-04-23T18:23:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b5a0f640 1 -- 172.21.15.37:0/2077779173 learned_addr learned my addr 172.21.15.37:0/2077779173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b520e640 1 -- 172.21.15.37:0/2077779173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 msgr2=0x55d8c9d62cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b520e640 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d62cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b520e640 1 -- 172.21.15.37:0/2077779173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 msgr2=0x55d8c9d7f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b520e640 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d7f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.684+0000 7f92b520e640 1 -- 172.21.15.37:0/2077779173 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c9d60e30 con 0x55d8c9d58620 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92b520e640 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d736a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92ac010850 tx=0x7f92ac0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c5f4045516dfce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f929affd640 1 -- 172.21.15.37:0/2077779173 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac029070 con 0x55d8c9d58620 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f929affd640 1 -- 172.21.15.37:0/2077779173 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ac0055a0 con 0x55d8c9d58620 2024-04-23T18:23:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f929affd640 1 -- 172.21.15.37:0/2077779173 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac003850 con 0x55d8c9d58620 2024-04-23T18:23:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 -- 172.21.15.37:0/2077779173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 msgr2=0x55d8c9d736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d736a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92ac010850 tx=0x7f92ac0057f0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 -- 172.21.15.37:0/2077779173 shutdown_connections 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d7f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58f40 0x55d8c9d62cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 --2- 172.21.15.37:0/2077779173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58620 0x55d8c9d736a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 -- 172.21.15.37:0/2077779173 >> 172.21.15.37:0/2077779173 conn(0x55d8c9d54b30 msgr2=0x55d8c9d5a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.685+0000 7f92ba186740 1 -- 172.21.15.37:0/2077779173 shutdown_connections 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 -- 172.21.15.37:0/2077779173 wait complete. 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 -- start start 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 Processor -- start 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58620 0x55d8c9d6fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58f40 0x55d8c9d700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01a9f40 con 0x55d8c9d63800 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01ab3b0 con 0x55d8c9d58620 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01aa610 con 0x55d8c9d58f40 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92ba186740 10 client.0 _ll_register_callbacks cb 0x7f92ac012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92b520e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58620 0x55d8c9d6fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92b5a0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92b4a0d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58f40 0x55d8c9d700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92b520e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58620 0x55d8c9d6fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:50576/0 (socket says 172.21.15.37:50576) 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.686+0000 7f92b520e640 1 -- 172.21.15.37:0/4195289780 learned_addr learned my addr 172.21.15.37:0/4195289780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b4a0d640 1 -- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58620 msgr2=0x55d8c9d6fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58620 0x55d8c9d6fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b4a0d640 1 -- 172.21.15.37:0/4195289780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 msgr2=0x55d8c9d6b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d6b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b4a0d640 1 -- 172.21.15.37:0/4195289780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c9d6bb00 con 0x55d8c9d58f40 2024-04-23T18:23:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b5a0f640 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92b4a0d640 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58f40 0x55d8c9d700f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92a00ee270 tx=0x7f92a002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52885b5aa22b12ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f9298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f9298ff9640 1 -- 172.21.15.37:0/4195289780 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0049040 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f9298ff9640 1 -- 172.21.15.37:0/4195289780 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f92a00bae70 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ac0055a0 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f9298ff9640 1 -- 172.21.15.37:0/4195289780 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92a0003d30 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92ba186740 10 client.25396 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.687+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8c9dddee0 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f9298ff9640 1 -- 172.21.15.37:0/4195289780 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ac0055a0 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f9298ff9640 1 -- 172.21.15.37:0/4195289780 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f92a00bcdd0 con 0x55d8c9d58f40 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92ba186740 10 client.25396 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92ba186740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92ba186740 20 client.25396 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92ba186740 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9dde5c0 0x55d8c9de09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92ba186740 4 client.25396 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:23:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8c9cc5200 con 0x55d8c9dde5c0 2024-04-23T18:23:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92b520e640 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9dde5c0 0x55d8c9de09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f92b520e640 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9dde5c0 0x55d8c9de09e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4793e2615dc1ee2 server_cookie=c23b3315f0441c6f in_seq=0 out_seq=0 2024-04-23T18:23:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.688+0000 7f9298ff9640 10 client.25396 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f9298ff9640 1 -- 172.21.15.37:0/4195289780 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8c9cc5200 con 0x55d8c9dde5c0 2024-04-23T18:23:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f9298ff9640 10 client.25396 handle_command_reply: tid=0 2024-04-23T18:23:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 client.25396 shutdown 2024-04-23T18:23:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58f40 msgr2=0x55d8c9d700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58f40 0x55d8c9d700f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92a00ee270 tx=0x7f92a002a410 comp rx=0 tx=0).stop 2024-04-23T18:23:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 shutdown_connections 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d8c9dde5c0 0x55d8c9de09e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8c9d63800 0x55d8c9d6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d8c9d58f40 0x55d8c9d700f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f9298ff9640 0 client.25396 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 --2- 172.21.15.37:0/4195289780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8c9d58620 0x55d8c9d6fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 >> 172.21.15.37:0/4195289780 conn(0x55d8c9d54b30 msgr2=0x55d8c9d5b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 shutdown_connections 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 1 -- 172.21.15.37:0/4195289780 wait complete. 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.734+0000 7f92ba186740 20 client.25396 trim_cache size 0 max 16384 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:23:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:23:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:23:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:23:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f929c0a3dc0 msgr2=0x7f929c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f929c0a3dc0 0x7f929c0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92a801e000 tx=0x7f92a8020000 comp rx=0 tx=0).stop 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 msgr2=0x7f92b0198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 0x7f92b0198c10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f92a0032fd0 tx=0x7f92a0031a20 comp rx=0 tx=0).stop 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 shutdown_connections 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f929c0a3dc0 0x7f929c0a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b019cf90 0x7f92b019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0106170 0x7f92b0198c10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 --2- 172.21.15.37:0/3692895964 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b01010f0 0x7f92b01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 >> 172.21.15.37:0/3692895964 conn(0x7f92b0077fd0 msgr2=0x7f92b00ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 shutdown_connections 2024-04-23T18:23:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:10.735+0000 7f92b7472640 1 -- 172.21.15.37:0/3692895964 wait complete. 2024-04-23T18:23:10.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 169509 12.41 MB/sec warmup 75 sec latency 16.117 ms 2024-04-23T18:23:11.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 171175 12.36 MB/sec warmup 76 sec latency 14.590 ms 2024-04-23T18:23:12.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:12 smithi172 ceph-mon[32456]: pgmap v85: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-04-23T18:23:12.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:12 smithi173 ceph-mon[32906]: pgmap v85: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-04-23T18:23:12.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:12 smithi037 ceph-mon[29404]: pgmap v85: 97 pgs: 97 active+clean; 97 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 308 op/s 2024-04-23T18:23:12.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 174030 12.44 MB/sec warmup 77 sec latency 12.606 ms 2024-04-23T18:23:13.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 175345 12.35 MB/sec warmup 78 sec latency 24.796 ms 2024-04-23T18:23:14.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:14 smithi172 ceph-mon[32456]: pgmap v86: 97 pgs: 97 active+clean; 98 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-04-23T18:23:14.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:14 smithi173 ceph-mon[32906]: pgmap v86: 97 pgs: 97 active+clean; 98 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-04-23T18:23:14.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:14 smithi037 ceph-mon[29404]: pgmap v86: 97 pgs: 97 active+clean; 98 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-04-23T18:23:14.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 177754 12.37 MB/sec warmup 79 sec latency 12.618 ms 2024-04-23T18:23:15.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 179851 12.31 MB/sec warmup 80 sec latency 17.175 ms 2024-04-23T18:23:16.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:16 smithi172 ceph-mon[32456]: pgmap v87: 97 pgs: 97 active+clean; 98 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-04-23T18:23:16.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:16 smithi173 ceph-mon[32906]: pgmap v87: 97 pgs: 97 active+clean; 98 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-04-23T18:23:16.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:16 smithi037 ceph-mon[29404]: pgmap v87: 97 pgs: 97 active+clean; 98 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-04-23T18:23:16.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 181532 12.30 MB/sec warmup 81 sec latency 16.390 ms 2024-04-23T18:23:17.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 184276 12.33 MB/sec warmup 82 sec latency 15.452 ms 2024-04-23T18:23:18.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:18 smithi172 ceph-mon[32456]: pgmap v88: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 406 op/s 2024-04-23T18:23:18.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:18 smithi173 ceph-mon[32906]: pgmap v88: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 406 op/s 2024-04-23T18:23:18.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:18 smithi037 ceph-mon[29404]: pgmap v88: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 406 op/s 2024-04-23T18:23:18.810 INFO:tasks.workunit.client.0.smithi037.stdout: 1 185246 12.24 MB/sec warmup 83 sec latency 33.434 ms 2024-04-23T18:23:19.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 188161 12.30 MB/sec warmup 84 sec latency 18.474 ms 2024-04-23T18:23:20.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:20 smithi172 ceph-mon[32456]: pgmap v89: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-04-23T18:23:20.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:20 smithi173 ceph-mon[32906]: pgmap v89: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-04-23T18:23:20.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:20 smithi037 ceph-mon[29404]: pgmap v89: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-04-23T18:23:20.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 189458 12.23 MB/sec warmup 85 sec latency 14.665 ms 2024-04-23T18:23:21.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 191839 12.25 MB/sec warmup 86 sec latency 13.294 ms 2024-04-23T18:23:22.406 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:22 smithi173 ceph-mon[32906]: pgmap v90: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-04-23T18:23:22.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:22 smithi172 ceph-mon[32456]: pgmap v90: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-04-23T18:23:22.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:22 smithi037 ceph-mon[29404]: pgmap v90: 97 pgs: 97 active+clean; 104 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-04-23T18:23:22.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 193610 12.18 MB/sec warmup 87 sec latency 14.788 ms 2024-04-23T18:23:23.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 195551 12.19 MB/sec warmup 88 sec latency 40.442 ms 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/2223410765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc1010f0 msgr2=0x7fe4dc1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/2223410765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc1010f0 0x7fe4dc1014d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009900 tx=0x7fe4d802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/2223410765 shutdown_connections 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/2223410765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc106170 0x7fe4dc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/2223410765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/2223410765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc1010f0 0x7fe4dc1014d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/2223410765 >> 172.21.15.37:0/2223410765 conn(0x7fe4dc077fd0 msgr2=0x7fe4dc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/2223410765 shutdown_connections 2024-04-23T18:23:23.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.826+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/2223410765 wait complete. 2024-04-23T18:23:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 Processor -- start 2024-04-23T18:23:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 -- start start 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1010f0 0x7fe4dc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 0x7fe4dc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4dc19cf70 0x7fe4dc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1110b0 con 0x7fe4dc106170 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc110f30 con 0x7fe4dc1010f0 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e3bcd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111230 con 0x7fe4dc19cf70 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 0x7fe4dc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 0x7fe4dc198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41344/0 (socket says 172.21.15.37:41344) 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 -- 172.21.15.37:0/4088115424 learned_addr learned my addr 172.21.15.37:0/4088115424 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e216a640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4dc19cf70 0x7fe4dc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1969640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1010f0 0x7fe4dc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 -- 172.21.15.37:0/4088115424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4dc19cf70 msgr2=0x7fe4dc19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4dc19cf70 0x7fe4dc19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 -- 172.21.15.37:0/4088115424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1010f0 msgr2=0x7fe4dc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1010f0 0x7fe4dc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 -- 172.21.15.37:0/4088115424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc19da60 con 0x7fe4dc106170 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.827+0000 7fe4e1168640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 0x7fe4dc198bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e9c0 tx=0x7fe4cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.828+0000 7fe4d2ffd640 1 -- 172.21.15.37:0/4088115424 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00cd80 con 0x7fe4dc106170 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.828+0000 7fe4d2ffd640 1 -- 172.21.15.37:0/4088115424 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc0054d0 con 0x7fe4dc106170 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.828+0000 7fe4d2ffd640 1 -- 172.21.15.37:0/4088115424 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc010430 con 0x7fe4dc106170 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.828+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc19dd50 con 0x7fe4dc106170 2024-04-23T18:23:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.828+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a5590 con 0x7fe4dc106170 2024-04-23T18:23:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.830+0000 7fe4d2ffd640 1 -- 172.21.15.37:0/4088115424 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4cc004fd0 con 0x7fe4dc106170 2024-04-23T18:23:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.830+0000 7fe4e69c1740 1 Processor -- start 2024-04-23T18:23:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 -- start start 2024-04-23T18:23:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf163ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4d2ffd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe4c80a3e40 0x7fe4c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163bf40 0x560bf16462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf1646800 0x560bf1648bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560bf15a0970 con 0x560bf163bf40 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bf153d4a0 con 0x560bf1646800 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560bf159c930 con 0x560bf163b620 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf163ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e1969640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe4c80a3e40 0x7fe4c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e1168640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163bf40 0x560bf16462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf163ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:46888/0 (socket says 172.21.15.37:46888) 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 -- 172.21.15.37:0/2066766114 learned_addr learned my addr 172.21.15.37:0/2066766114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf1646800 0x560bf1648bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4d2ffd640 1 -- 172.21.15.37:0/4088115424 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe4cc0d0bc0 con 0x7fe4dc106170 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 -- 172.21.15.37:0/2066766114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf1646800 msgr2=0x560bf1648bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf1646800 0x560bf1648bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 -- 172.21.15.37:0/2066766114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163bf40 msgr2=0x560bf16462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163bf40 0x560bf16462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.831+0000 7fe4e216a640 1 -- 172.21.15.37:0/2066766114 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf12b3210 con 0x560bf163b620 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.832+0000 7fe4e216a640 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf163ba00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005c20 tx=0x7fe4d40099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b2dbf49a42676d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.832+0000 7fe4e1969640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe4c80a3e40 0x7fe4c80a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8031440 tx=0x7fe4d8005b50 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.832+0000 7fe4c7fff640 1 -- 172.21.15.37:0/2066766114 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4003970 con 0x560bf163b620 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.832+0000 7fe4c7fff640 1 -- 172.21.15.37:0/2066766114 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d4010770 con 0x560bf163b620 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.832+0000 7fe4c7fff640 1 -- 172.21.15.37:0/2066766114 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40042e0 con 0x560bf163b620 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 -- 172.21.15.37:0/2066766114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 msgr2=0x560bf163ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf163ba00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4005c20 tx=0x7fe4d40099f0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 -- 172.21.15.37:0/2066766114 shutdown_connections 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf1646800 0x560bf1648bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163bf40 0x560bf16462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/2066766114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf163ba00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 -- 172.21.15.37:0/2066766114 >> 172.21.15.37:0/2066766114 conn(0x560bf1637b30 msgr2=0x560bf1637f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 -- 172.21.15.37:0/2066766114 shutdown_connections 2024-04-23T18:23:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 -- 172.21.15.37:0/2066766114 wait complete. 2024-04-23T18:23:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 -- start start 2024-04-23T18:23:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.833+0000 7fe4e69c1740 1 Processor -- start 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 0x560bf16461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf16434c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf1646800 0x560bf1643ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560bf1649430 con 0x560bf163b620 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bf153b0b0 con 0x560bf163bf40 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560bf16495b0 con 0x560bf1646800 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 0x560bf16461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e69c1740 10 client.0 _ll_register_callbacks cb 0x560bf1528d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 0x560bf16461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41352/0 (socket says 172.21.15.37:41352) 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 -- 172.21.15.37:0/341639371 learned_addr learned my addr 172.21.15.37:0/341639371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1168640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf16434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e216a640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf1646800 0x560bf1643ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 -- 172.21.15.37:0/341639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf1646800 msgr2=0x560bf1643ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf1646800 0x560bf1643ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 -- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 msgr2=0x560bf16434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf16434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 -- 172.21.15.37:0/341639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf137f620 con 0x560bf163b620 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1168640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf16434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4e1969640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 0x560bf16461d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8033000 tx=0x7fe4d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822b1efcb1547b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.834+0000 7fe4c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4c57fa640 1 -- 172.21.15.37:0/341639371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d804c020 con 0x560bf163b620 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4c57fa640 1 -- 172.21.15.37:0/341639371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe4d8004040 con 0x560bf163b620 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4010770 con 0x560bf163b620 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4c57fa640 1 -- 172.21.15.37:0/341639371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4d8057050 con 0x560bf163b620 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 10 client.15852 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560bf16553c0 con 0x560bf163b620 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4c57fa640 1 -- 172.21.15.37:0/341639371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d4010770 con 0x560bf163b620 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4c57fa640 1 -- 172.21.15.37:0/341639371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fe4d8004870 con 0x560bf163b620 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 20 client.15852 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16568d0 0x560bf1658cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 4 client.15852 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bf15ae390 con 0x560bf16568d0 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.835+0000 7fe4e1168640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16568d0 0x560bf1658cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.836+0000 7fe4e1168640 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16568d0 0x560bf1658cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22838c134ab4dd31 server_cookie=ad25493621408a67 in_seq=0 out_seq=0 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.836+0000 7fe4c57fa640 10 client.15852 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.836+0000 7fe4c57fa640 1 -- 172.21.15.37:0/341639371 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bf15ae390 con 0x560bf16568d0 2024-04-23T18:23:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.836+0000 7fe4c57fa640 10 client.15852 handle_command_reply: tid=0 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.836+0000 7fe4e69c1740 1 client.15852 shutdown 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 msgr2=0x560bf16461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 0x560bf16461d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8033000 tx=0x7fe4d8009260 comp rx=0 tx=0).stop 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 shutdown_connections 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16568d0 0x560bf1658cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf1646800 0x560bf1643ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf16434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4c57fa640 0 client.15852 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/341639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf163b620 0x560bf16461d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 >> 172.21.15.37:0/341639371 conn(0x560bf1637b30 msgr2=0x560bf163d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 shutdown_connections 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 1 -- 172.21.15.37:0/341639371 wait complete. 2024-04-23T18:23:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.837+0000 7fe4e69c1740 20 client.15852 trim_cache size 0 max 16384 2024-04-23T18:23:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.852+0000 7fe4e69c1740 1 Processor -- start 2024-04-23T18:23:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.852+0000 7fe4e69c1740 1 -- start start 2024-04-23T18:23:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf16522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf1636b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf165e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1b04f0 con 0x560bf1646800 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1add60 con 0x560bf163bf40 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1abbb0 con 0x560bf163b620 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf165e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf165e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41358/0 (socket says 172.21.15.37:41358) 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e1969640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf16522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 -- 172.21.15.37:0/522607702 learned_addr learned my addr 172.21.15.37:0/522607702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e1168640 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf1636b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 -- 172.21.15.37:0/522607702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 msgr2=0x560bf16522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf16522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 -- 172.21.15.37:0/522607702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 msgr2=0x560bf1636b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf1636b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 -- 172.21.15.37:0/522607702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf16424b0 con 0x560bf1646800 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4e216a640 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf165e590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4003b30 tx=0x7fe4d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4126ebd3c02654d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4c77fe640 1 -- 172.21.15.37:0/522607702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4010740 con 0x560bf1646800 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4c77fe640 1 -- 172.21.15.37:0/522607702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d4016040 con 0x560bf1646800 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.853+0000 7fe4c77fe640 1 -- 172.21.15.37:0/522607702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4005370 con 0x560bf1646800 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 -- 172.21.15.37:0/522607702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 msgr2=0x560bf165e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf165e590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4003b30 tx=0x7fe4d4005fd0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 -- 172.21.15.37:0/522607702 shutdown_connections 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf165e590 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf1636b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/522607702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf16522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 -- 172.21.15.37:0/522607702 >> 172.21.15.37:0/522607702 conn(0x560bf1637b30 msgr2=0x560bf1636420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 -- 172.21.15.37:0/522607702 shutdown_connections 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 -- 172.21.15.37:0/522607702 wait complete. 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 -- start start 2024-04-23T18:23:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.854+0000 7fe4e69c1740 1 Processor -- start 2024-04-23T18:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf164c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf164c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf164cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ac220 con 0x560bf1646800 2024-04-23T18:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ad690 con 0x560bf163bf40 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ac8f0 con 0x560bf163b620 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e69c1740 10 client.0 _ll_register_callbacks cb 0x7fe4d8032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf164cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e1969640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf164c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf164cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41362/0 (socket says 172.21.15.37:41362) 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e1969640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf164c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:46920/0 (socket says 172.21.15.37:46920) 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 -- 172.21.15.37:0/1015419184 learned_addr learned my addr 172.21.15.37:0/1015419184 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e1168640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf164c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 -- 172.21.15.37:0/1015419184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 msgr2=0x560bf164c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf164c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 -- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 msgr2=0x560bf164c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf164c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e216a640 1 -- 172.21.15.37:0/1015419184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf1643e40 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.855+0000 7fe4e1168640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf164c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e216a640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf164cbd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40129f0 tx=0x7fe4d4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb68c7fa0ada153 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4c57fa640 1 -- 172.21.15.37:0/1015419184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4009d40 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4c57fa640 1 -- 172.21.15.37:0/1015419184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe4d40050b0 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4016040 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4c57fa640 1 -- 172.21.15.37:0/1015419184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4d403f050 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560bf16bce50 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4c57fa640 1 -- 172.21.15.37:0/1015419184 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d4016040 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4c57fa640 1 -- 172.21.15.37:0/1015419184 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fe4d400f040 con 0x560bf1646800 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 20 client.15858 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16bd320 0x560bf16bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 4 client.15858 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:23:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.856+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bf15a8200 con 0x560bf16bd320 2024-04-23T18:23:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.857+0000 7fe4e1969640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16bd320 0x560bf16bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.857+0000 7fe4e1969640 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16bd320 0x560bf16bf740 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffede46db6ac2a74 server_cookie=ad18762f8d8f36f9 in_seq=0 out_seq=0 2024-04-23T18:23:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.857+0000 7fe4c57fa640 10 client.15858 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.893+0000 7fe4c57fa640 1 -- 172.21.15.37:0/1015419184 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bf15a8200 con 0x560bf16bd320 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.893+0000 7fe4c57fa640 10 client.15858 handle_command_reply: tid=0 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 client.15858 shutdown 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 msgr2=0x560bf164cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf164cbd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40129f0 tx=0x7fe4d4009670 comp rx=0 tx=0).stop 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 shutdown_connections 2024-04-23T18:23:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560bf16bd320 0x560bf16bf740 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560bf1646800 0x560bf164cbd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bf163bf40 0x560bf164c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4c57fa640 0 client.15858 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 --2- 172.21.15.37:0/1015419184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560bf163b620 0x560bf164c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 >> 172.21.15.37:0/1015419184 conn(0x560bf1637b30 msgr2=0x560bf1636cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 shutdown_connections 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 1 -- 172.21.15.37:0/1015419184 wait complete. 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e69c1740 20 client.15858 trim_cache size 0 max 16384 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:23:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:23:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:23:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:23:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe4c80a3e40 msgr2=0x7fe4c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe4c80a3e40 0x7fe4c80a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8031440 tx=0x7fe4d8005b50 comp rx=0 tx=0).stop 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 msgr2=0x7fe4dc198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.894+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 0x7fe4dc198bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e9c0 tx=0x7fe4cc00ee80 comp rx=0 tx=0).stop 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 shutdown_connections 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe4c80a3e40 0x7fe4c80a6300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4dc19cf70 0x7fe4dc19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc106170 0x7fe4dc198bf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 --2- 172.21.15.37:0/4088115424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1010f0 0x7fe4dc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 >> 172.21.15.37:0/4088115424 conn(0x7fe4dc077fd0 msgr2=0x7fe4dc0699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 shutdown_connections 2024-04-23T18:23:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:23.895+0000 7fe4e3bcd640 1 -- 172.21.15.37:0/4088115424 wait complete. 2024-04-23T18:23:24.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:24 smithi172 ceph-mon[32456]: pgmap v91: 97 pgs: 97 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 404 op/s 2024-04-23T18:23:24.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:24 smithi173 ceph-mon[32906]: pgmap v91: 97 pgs: 97 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 404 op/s 2024-04-23T18:23:24.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:24 smithi037 ceph-mon[29404]: pgmap v91: 97 pgs: 97 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 404 op/s 2024-04-23T18:23:24.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 198074 12.19 MB/sec warmup 89 sec latency 26.636 ms 2024-04-23T18:23:25.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 199731 12.15 MB/sec warmup 90 sec latency 15.384 ms 2024-04-23T18:23:26.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:26 smithi172 ceph-mon[32456]: pgmap v92: 97 pgs: 97 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-04-23T18:23:26.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:26 smithi173 ceph-mon[32906]: pgmap v92: 97 pgs: 97 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-04-23T18:23:26.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:26 smithi037 ceph-mon[29404]: pgmap v92: 97 pgs: 97 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-04-23T18:23:26.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 202479 12.20 MB/sec warmup 91 sec latency 12.168 ms 2024-04-23T18:23:27.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 204480 12.15 MB/sec warmup 92 sec latency 14.774 ms 2024-04-23T18:23:28.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:28 smithi172 ceph-mon[32456]: pgmap v93: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 402 op/s 2024-04-23T18:23:28.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:28 smithi173 ceph-mon[32906]: pgmap v93: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 402 op/s 2024-04-23T18:23:28.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:28 smithi037 ceph-mon[29404]: pgmap v93: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 402 op/s 2024-04-23T18:23:28.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 206195 12.15 MB/sec warmup 93 sec latency 40.144 ms 2024-04-23T18:23:29.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 208621 12.15 MB/sec warmup 94 sec latency 25.961 ms 2024-04-23T18:23:30.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:30 smithi172 ceph-mon[32456]: pgmap v94: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-04-23T18:23:30.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:30 smithi173 ceph-mon[32906]: pgmap v94: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-04-23T18:23:30.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:30 smithi037 ceph-mon[29404]: pgmap v94: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-04-23T18:23:30.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 210191 12.10 MB/sec warmup 95 sec latency 14.867 ms 2024-04-23T18:23:31.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 213031 12.17 MB/sec warmup 96 sec latency 12.475 ms 2024-04-23T18:23:32.421 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:32 smithi037 ceph-mon[29404]: pgmap v95: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-04-23T18:23:32.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:32 smithi172 ceph-mon[32456]: pgmap v95: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-04-23T18:23:32.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:32 smithi173 ceph-mon[32906]: pgmap v95: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-04-23T18:23:32.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 214479 12.10 MB/sec warmup 97 sec latency 15.925 ms 2024-04-23T18:23:33.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 216789 12.12 MB/sec warmup 98 sec latency 40.235 ms 2024-04-23T18:23:34.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:34 smithi172 ceph-mon[32456]: pgmap v96: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 402 op/s 2024-04-23T18:23:34.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:34 smithi173 ceph-mon[32906]: pgmap v96: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 402 op/s 2024-04-23T18:23:34.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:34 smithi037 ceph-mon[29404]: pgmap v96: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 402 op/s 2024-04-23T18:23:34.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 218830 12.07 MB/sec warmup 99 sec latency 22.806 ms 2024-04-23T18:23:35.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 220577 12.07 MB/sec warmup 100 sec latency 16.472 ms 2024-04-23T18:23:36.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:36 smithi172 ceph-mon[32456]: pgmap v97: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 280 op/s 2024-04-23T18:23:36.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:36 smithi173 ceph-mon[32906]: pgmap v97: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 280 op/s 2024-04-23T18:23:36.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:36 smithi037 ceph-mon[29404]: pgmap v97: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 280 op/s 2024-04-23T18:23:36.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223474 12.12 MB/sec warmup 101 sec latency 14.110 ms 2024-04-23T18:23:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 -- 172.21.15.37:0/2193194727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f010c540 msgr2=0x7f05f010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 --2- 172.21.15.37:0/2193194727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f010c540 0x7f05f010e9b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009ca0 tx=0x7f05e802a950 comp rx=0 tx=0).stop 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 -- 172.21.15.37:0/2193194727 shutdown_connections 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 --2- 172.21.15.37:0/2193194727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f010c540 0x7f05f010e9b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 --2- 172.21.15.37:0/2193194727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05f0103b20 0x7f05f010c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 --2- 172.21.15.37:0/2193194727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f0103200 0x7f05f01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 -- 172.21.15.37:0/2193194727 >> 172.21.15.37:0/2193194727 conn(0x7f05f0077fd0 msgr2=0x7f05f00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 -- 172.21.15.37:0/2193194727 shutdown_connections 2024-04-23T18:23:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 -- 172.21.15.37:0/2193194727 wait complete. 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.984+0000 7f05f622c640 1 Processor -- start 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 -- start start 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05f0103200 0x7f05f0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103b20 0x7f05f0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f010c540 0x7f05f019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0112180 con 0x7f05f0103b20 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0112000 con 0x7f05f0103200 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05f622c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0112300 con 0x7f05f010c540 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103b20 0x7f05f0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103b20 0x7f05f0198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49428/0 (socket says 172.21.15.37:49428) 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05eeffd640 1 -- 172.21.15.37:0/4288961014 learned_addr learned my addr 172.21.15.37:0/4288961014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f010c540 0x7f05f019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05f0103200 0x7f05f0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 -- 172.21.15.37:0/4288961014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05f0103200 msgr2=0x7f05f0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05f0103200 0x7f05f0198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 -- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103b20 msgr2=0x7f05f0198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103b20 0x7f05f0198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 -- 172.21.15.37:0/4288961014 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f019d620 con 0x7f05f010c540 2024-04-23T18:23:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.985+0000 7f05effff640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f010c540 0x7f05f019cef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005cc0 tx=0x7f05e802f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.986+0000 7f05ecff9640 1 -- 172.21.15.37:0/4288961014 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8034040 con 0x7f05f010c540 2024-04-23T18:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.986+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f019d8b0 con 0x7f05f010c540 2024-04-23T18:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.986+0000 7f05ecff9640 1 -- 172.21.15.37:0/4288961014 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e80308f0 con 0x7f05f010c540 2024-04-23T18:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.986+0000 7f05ecff9640 1 -- 172.21.15.37:0/4288961014 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e80045e0 con 0x7f05f010c540 2024-04-23T18:23:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.986+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01a51a0 con 0x7f05f010c540 2024-04-23T18:23:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.988+0000 7f05ecff9640 1 -- 172.21.15.37:0/4288961014 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05e8037020 con 0x7f05f010c540 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.988+0000 7f05f9016740 1 Processor -- start 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.988+0000 7f05f9016740 1 -- start start 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05f9016740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0620 0x55650daf0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05f9016740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650dafb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05f9016740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650dafb800 0x55650dafdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05f9016740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55650da55970 con 0x55650daf0f40 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05f9016740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55650d9f24a0 con 0x55650daf0620 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05f9016740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55650da51930 con 0x55650dafb800 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650dafb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05ecff9640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f05d80a3dc0 0x7f05d80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0620 0x55650daf0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650dafb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49436/0 (socket says 172.21.15.37:49436) 2024-04-23T18:23:36.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05ef7fe640 1 -- 172.21.15.37:0/1488326310 learned_addr learned my addr 172.21.15.37:0/1488326310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650dafb800 0x55650dafdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05effff640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f05d80a3dc0 0x7f05d80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05ecff9640 1 -- 172.21.15.37:0/4288961014 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05e80f1450 con 0x7f05f010c540 2024-04-23T18:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 -- 172.21.15.37:0/1488326310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650dafb800 msgr2=0x55650dafdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650dafb800 0x55650dafdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 -- 172.21.15.37:0/1488326310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 msgr2=0x55650dafb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650dafb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 -- 172.21.15.37:0/1488326310 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650d768210 con 0x55650daf0620 2024-04-23T18:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650dafb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0620 0x55650daf0a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b770 tx=0x7f05e000b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1fbaf5e7ddd02 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05effff640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f05d80a3dc0 0x7f05d80a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05e80bd190 tx=0x7f05e802a950 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.989+0000 7f05d5ffb640 1 -- 172.21.15.37:0/1488326310 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0012d00 con 0x55650daf0620 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05d5ffb640 1 -- 172.21.15.37:0/1488326310 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0005020 con 0x55650daf0620 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05d5ffb640 1 -- 172.21.15.37:0/1488326310 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0004a00 con 0x55650daf0620 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 -- 172.21.15.37:0/1488326310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0620 msgr2=0x55650daf0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0620 0x55650daf0a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b770 tx=0x7f05e000b700 comp rx=0 tx=0).stop 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 -- 172.21.15.37:0/1488326310 shutdown_connections 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650dafb800 0x55650dafdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650dafb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 --2- 172.21.15.37:0/1488326310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0620 0x55650daf0a00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 -- 172.21.15.37:0/1488326310 >> 172.21.15.37:0/1488326310 conn(0x55650daecb30 msgr2=0x55650daecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 -- 172.21.15.37:0/1488326310 shutdown_connections 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 -- 172.21.15.37:0/1488326310 wait complete. 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.990+0000 7f05f9016740 1 -- start start 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 Processor -- start 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650dafb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650daf8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650dafb800 0x55650daf8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55650dafe430 con 0x55650daf0f40 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55650d9f00b0 con 0x55650dafb800 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55650dafe5b0 con 0x55650daf0620 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05f9016740 10 client.0 _ll_register_callbacks cb 0x55650d9ddd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650daf8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650dafb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650daf8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49448/0 (socket says 172.21.15.37:49448) 2024-04-23T18:23:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 -- 172.21.15.37:0/3575163151 learned_addr learned my addr 172.21.15.37:0/3575163151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05eeffd640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650dafb800 0x55650daf8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 -- 172.21.15.37:0/3575163151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 msgr2=0x55650dafb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650dafb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 -- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650dafb800 msgr2=0x55650daf8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650dafb800 0x55650daf8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05ef7fe640 1 -- 172.21.15.37:0/3575163151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650d834620 con 0x55650daf0f40 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.991+0000 7f05eeffd640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650dafb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650daf8490 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f05e4012740 tx=0x7f05e4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ef84d116350317 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ceffd640 1 -- 172.21.15.37:0/3575163151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e400f920 con 0x55650daf0f40 2024-04-23T18:23:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e0005020 con 0x55650daf0f40 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ceffd640 1 -- 172.21.15.37:0/3575163151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05e400fac0 con 0x55650daf0f40 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ceffd640 1 -- 172.21.15.37:0/3575163151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05e400fcc0 con 0x55650daf0f40 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05f9016740 10 client.15870 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55650db0a490 con 0x55650daf0f40 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ceffd640 1 -- 172.21.15.37:0/3575163151 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0005020 con 0x55650daf0f40 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.992+0000 7f05ceffd640 1 -- 172.21.15.37:0/3575163151 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f05e401c420 con 0x55650daf0f40 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05f9016740 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05f9016740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05f9016740 20 client.15870 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05f9016740 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db0b9c0 0x55650db0ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05f9016740 4 client.15870 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:23:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55650da63390 con 0x55650db0b9c0 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05eeffd640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db0b9c0 0x55650db0ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05ceffd640 10 client.15870 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.993+0000 7f05eeffd640 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db0b9c0 0x55650db0ddb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f541b92406c0be1d server_cookie=bac20700c5ae560c in_seq=0 out_seq=0 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05ceffd640 1 -- 172.21.15.37:0/3575163151 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55650da63390 con 0x55650db0b9c0 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05ceffd640 10 client.15870 handle_command_reply: tid=0 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 client.15870 shutdown 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 msgr2=0x55650daf8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650daf8490 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f05e4012740 tx=0x7f05e4012c00 comp rx=0 tx=0).stop 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 shutdown_connections 2024-04-23T18:23:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db0b9c0 0x55650db0ddb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650dafb800 0x55650daf8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650daf0f40 0x55650daf8490 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05ceffd640 0 client.15870 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 --2- 172.21.15.37:0/3575163151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650dafb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 >> 172.21.15.37:0/3575163151 conn(0x55650daecb30 msgr2=0x55650daf28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 shutdown_connections 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 1 -- 172.21.15.37:0/3575163151 wait complete. 2024-04-23T18:23:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:36.994+0000 7f05f9016740 20 client.15870 trim_cache size 0 max 16384 2024-04-23T18:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 Processor -- start 2024-04-23T18:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 -- start start 2024-04-23T18:23:37.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650dafacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650db13620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b0460 con 0x55650dafb800 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01adcd0 con 0x55650daf0f40 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05f9016740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01abb20 con 0x55650daf0620 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650db13620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650db13620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49456/0 (socket says 172.21.15.37:49456) 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 -- 172.21.15.37:0/1168019287 learned_addr learned my addr 172.21.15.37:0/1168019287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650dafacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 -- 172.21.15.37:0/1168019287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 msgr2=0x55650db072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 -- 172.21.15.37:0/1168019287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 msgr2=0x55650dafacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.009+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650dafacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05eeffd640 1 -- 172.21.15.37:0/1168019287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650db0e120 con 0x55650dafb800 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05eeffd640 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650db13620 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f05e0012fd0 tx=0x7f05e0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b30b214c736a3cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05d57fa640 1 -- 172.21.15.37:0/1168019287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0005570 con 0x55650dafb800 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05d57fa640 1 -- 172.21.15.37:0/1168019287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e002e880 con 0x55650dafb800 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05d57fa640 1 -- 172.21.15.37:0/1168019287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0010e50 con 0x55650dafb800 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 -- 172.21.15.37:0/1168019287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 msgr2=0x55650db13620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650db13620 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f05e0012fd0 tx=0x7f05e0014040 comp rx=0 tx=0).stop 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 -- 172.21.15.37:0/1168019287 shutdown_connections 2024-04-23T18:23:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650db13620 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650dafacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 --2- 172.21.15.37:0/1168019287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 -- 172.21.15.37:0/1168019287 >> 172.21.15.37:0/1168019287 conn(0x55650daecb30 msgr2=0x55650daf3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.010+0000 7f05f9016740 1 -- 172.21.15.37:0/1168019287 shutdown_connections 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 -- 172.21.15.37:0/1168019287 wait complete. 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 -- start start 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 Processor -- start 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db03790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650db03cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650dafedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01ac190 con 0x55650dafb800 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01ad600 con 0x55650daf0f40 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01ac860 con 0x55650daf0620 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05f9016740 10 client.0 _ll_register_callbacks cb 0x7f05e4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650dafedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05eeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650dafedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49464/0 (socket says 172.21.15.37:49464) 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05eeffd640 1 -- 172.21.15.37:0/2395057231 learned_addr learned my addr 172.21.15.37:0/2395057231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650db03cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.011+0000 7f05eeffd640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db03790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05eeffd640 1 -- 172.21.15.37:0/2395057231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 msgr2=0x55650db03790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05eeffd640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db03790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05eeffd640 1 -- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 msgr2=0x55650db03cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05eeffd640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650db03cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05eeffd640 1 -- 172.21.15.37:0/2395057231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650daff5f0 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05eeffd640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650dafedc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f05e002eab0 tx=0x7f05e000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801e2b4df8e37bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05ceffd640 1 -- 172.21.15.37:0/2395057231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0033040 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05ceffd640 1 -- 172.21.15.37:0/2395057231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05e002e3d0 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e002e880 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05ceffd640 1 -- 172.21.15.37:0/2395057231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05e002e5d0 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05f9016740 10 client.15876 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55650db71a40 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.012+0000 7f05ceffd640 1 -- 172.21.15.37:0/2395057231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e002e880 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05ceffd640 1 -- 172.21.15.37:0/2395057231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f05e0039070 con 0x55650dafb800 2024-04-23T18:23:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05f9016740 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05f9016740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05f9016740 20 client.15876 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05f9016740 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db71f10 0x55650db74300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05f9016740 4 client.15876 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55650da5d200 con 0x55650db71f10 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db71f10 0x55650db74300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05ceffd640 10 client.15876 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.013+0000 7f05ef7fe640 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db71f10 0x55650db74300 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6948337e613a524e server_cookie=5cdfd2166135a3de in_seq=0 out_seq=0 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.053+0000 7f05ceffd640 1 -- 172.21.15.37:0/2395057231 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55650da5d200 con 0x55650db71f10 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.053+0000 7f05ceffd640 10 client.15876 handle_command_reply: tid=0 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.053+0000 7f05f9016740 1 client.15876 shutdown 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.053+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 msgr2=0x55650dafedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.053+0000 7f05f9016740 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650dafedc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f05e002eab0 tx=0x7f05e000b480 comp rx=0 tx=0).stop 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 shutdown_connections 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55650db71f10 0x55650db74300 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55650dafb800 0x55650dafedc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650daf0f40 0x55650db03cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 --2- 172.21.15.37:0/2395057231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55650daf0620 0x55650db03790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05ceffd640 0 client.15876 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 >> 172.21.15.37:0/2395057231 conn(0x55650daecb30 msgr2=0x55650daf31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 shutdown_connections 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 1 -- 172.21.15.37:0/2395057231 wait complete. 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f9016740 20 client.15876 trim_cache size 0 max 16384 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:23:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f05d80a3dc0 msgr2=0x7f05d80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f05d80a3dc0 0x7f05d80a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f05e80bd190 tx=0x7f05e802a950 comp rx=0 tx=0).stop 2024-04-23T18:23:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f010c540 msgr2=0x7f05f019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f010c540 0x7f05f019cef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005cc0 tx=0x7f05e802f3a0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 shutdown_connections 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f05d80a3dc0 0x7f05d80a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f010c540 0x7f05f019cef0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103b20 0x7f05f0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 --2- 172.21.15.37:0/4288961014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05f0103200 0x7f05f0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.054+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 >> 172.21.15.37:0/4288961014 conn(0x7f05f0077fd0 msgr2=0x7f05f00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.055+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 shutdown_connections 2024-04-23T18:23:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:37.055+0000 7f05f622c640 1 -- 172.21.15.37:0/4288961014 wait complete. 2024-04-23T18:23:37.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 224898 12.07 MB/sec warmup 102 sec latency 20.699 ms 2024-04-23T18:23:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:38 smithi172 ceph-mon[32456]: pgmap v98: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 399 op/s 2024-04-23T18:23:38.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:38 smithi173 ceph-mon[32906]: pgmap v98: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 399 op/s 2024-04-23T18:23:38.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:38 smithi037 ceph-mon[29404]: pgmap v98: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 399 op/s 2024-04-23T18:23:38.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227202 12.07 MB/sec warmup 103 sec latency 59.708 ms 2024-04-23T18:23:39.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:39 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:39.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:39 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:39 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:39.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:39 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:23:39.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:39 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:39.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:39 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:39 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:39.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:39 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:23:39.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:39 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:39.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:39 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:39 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:39.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:39 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:23:39.811 INFO:tasks.workunit.client.0.smithi037.stdout: 1 228466 12.02 MB/sec warmup 104 sec latency 42.731 ms 2024-04-23T18:23:40.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:40 smithi172 ceph-mon[32456]: pgmap v99: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-04-23T18:23:40.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:40 smithi173 ceph-mon[32906]: pgmap v99: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-04-23T18:23:40.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:40 smithi037 ceph-mon[29404]: pgmap v99: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-04-23T18:23:40.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 230922 12.03 MB/sec warmup 105 sec latency 11.847 ms 2024-04-23T18:23:41.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 232973 11.99 MB/sec warmup 106 sec latency 17.166 ms 2024-04-23T18:23:42.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:42 smithi172 ceph-mon[32456]: pgmap v100: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-04-23T18:23:42.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:42 smithi173 ceph-mon[32906]: pgmap v100: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-04-23T18:23:42.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:42 smithi037 ceph-mon[29404]: pgmap v100: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-04-23T18:23:42.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 234864 11.99 MB/sec warmup 107 sec latency 12.018 ms 2024-04-23T18:23:43.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 237523 12.01 MB/sec warmup 108 sec latency 35.874 ms 2024-04-23T18:23:44.537 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:23:44.537 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-04-23T18:23:44.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:44.560 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:44 smithi037 ceph-mon[29404]: pgmap v101: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 389 op/s 2024-04-23T18:23:44.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:44 smithi172 ceph-mon[32456]: pgmap v101: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 389 op/s 2024-04-23T18:23:44.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:44 smithi173 ceph-mon[32906]: pgmap v101: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 389 op/s 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 -- 172.21.15.37:0/1129008634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 msgr2=0x7f29b4102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 --2- 172.21.15.37:0/1129008634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b4102580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009900 tx=0x7f29ac02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 -- 172.21.15.37:0/1129008634 shutdown_connections 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 --2- 172.21.15.37:0/1129008634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4107270 0x7f29b410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 --2- 172.21.15.37:0/1129008634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b4102b50 0x7f29b4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 --2- 172.21.15.37:0/1129008634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b4102580 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 -- 172.21.15.37:0/1129008634 >> 172.21.15.37:0/1129008634 conn(0x7f29b4079350 msgr2=0x7f29b4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 -- 172.21.15.37:0/1129008634 shutdown_connections 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 -- 172.21.15.37:0/1129008634 wait complete. 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 Processor -- start 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.655+0000 7f29b9795640 1 -- start start 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b9795640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b9795640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b4107270 0x7f29b419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b9795640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4198590 0x7f29b4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b9795640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29b41121b0 con 0x7f29b41021a0 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b9795640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f29b4112030 con 0x7f29b4107270 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b9795640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29b4112330 con 0x7f29b4198590 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59764/0 (socket says 172.21.15.37:59764) 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 -- 172.21.15.37:0/211508091 learned_addr learned my addr 172.21.15.37:0/211508091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b27fc640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b4107270 0x7f29b419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b37fe640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4198590 0x7f29b4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 -- 172.21.15.37:0/211508091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4198590 msgr2=0x7f29b4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4198590 0x7f29b4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 -- 172.21.15.37:0/211508091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b4107270 msgr2=0x7f29b419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b4107270 0x7f29b419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 -- 172.21.15.37:0/211508091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b41990b0 con 0x7f29b41021a0 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b37fe640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4198590 0x7f29b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:44.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29b2ffd640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b419e3e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005940 tx=0x7f29ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:44.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.656+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac03f030 con 0x7f29b41021a0 2024-04-23T18:23:44.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.657+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b4199340 con 0x7f29b41021a0 2024-04-23T18:23:44.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.657+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac0044c0 con 0x7f29b41021a0 2024-04-23T18:23:44.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.657+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac035070 con 0x7f29b41021a0 2024-04-23T18:23:44.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.657+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b41a5540 con 0x7f29b41021a0 2024-04-23T18:23:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.658+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b4068d70 con 0x7f29b41021a0 2024-04-23T18:23:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.659+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29ac042030 con 0x7f29b41021a0 2024-04-23T18:23:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.660+0000 7f29abfff640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f29a00a3d20 0x7f29a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.660+0000 7f29b27fc640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f29a00a3d20 0x7f29a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.660+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f29ac0f0b10 con 0x7f29b41021a0 2024-04-23T18:23:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.660+0000 7f29b27fc640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f29a00a3d20 0x7f29a00a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f29b4101b70 tx=0x7f299c005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.665+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29ac0bcb50 con 0x7f29b41021a0 2024-04-23T18:23:44.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 239081 11.99 MB/sec warmup 109 sec latency 27.644 ms 2024-04-23T18:23:44.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.915+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29b4199ac0 con 0x7f29b41021a0 2024-04-23T18:23:44.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.916+0000 7f29abfff640 1 -- 172.21.15.37:0/211508091 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11878 (secure 0 0 0) 0x7f29ac0c0e50 con 0x7f29b41021a0 2024-04-23T18:23:44.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:23:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:22:58.540054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:23:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-04-23T18:23:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f29a00a3d20 msgr2=0x7f29a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f29a00a3d20 0x7f29a00a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f29b4101b70 tx=0x7f299c005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 msgr2=0x7f29b419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b419e3e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005940 tx=0x7f29ac0040c0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 shutdown_connections 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f29a00a3d20 0x7f29a00a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29b4198590 0x7f29b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b4107270 0x7f29b419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 --2- 172.21.15.37:0/211508091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29b41021a0 0x7f29b419e3e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 >> 172.21.15.37:0/211508091 conn(0x7f29b4079350 msgr2=0x7f29b4100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 shutdown_connections 2024-04-23T18:23:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:44.918+0000 7f29b9795640 1 -- 172.21.15.37:0/211508091 wait complete. 2024-04-23T18:23:44.923 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 80.35 --await-for 42.0' 2024-04-23T18:23:44.923 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 80.35 --await-for 42.0 2024-04-23T18:23:45.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/2204543105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 msgr2=0x7f7cdc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/2204543105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc10dd40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009ca0 tx=0x7f7cd402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/2204543105 shutdown_connections 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/2204543105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc10dd40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/2204543105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc101a50 0x7f7cdc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/2204543105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 0x7f7cdc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/2204543105 >> 172.21.15.37:0/2204543105 conn(0x7f7cdc077f50 msgr2=0x7f7cdc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/2204543105 shutdown_connections 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.088+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/2204543105 wait complete. 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 Processor -- start 2024-04-23T18:23:45.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 -- start start 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 0x7f7cdc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc19cfa0 0x7f7cdc19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc111130 con 0x7f7cdc106170 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc110fb0 con 0x7f7cdc19cfa0 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.089+0000 7f7ce1e2a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1112b0 con 0x7f7cdc1010a0 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 0x7f7cdc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59786/0 (socket says 172.21.15.37:59786) 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc19cfa0 0x7f7cdc19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 -- 172.21.15.37:0/166483749 learned_addr learned my addr 172.21.15.37:0/166483749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 -- 172.21.15.37:0/166483749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 msgr2=0x7f7cdc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 0x7f7cdc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 -- 172.21.15.37:0/166483749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc19cfa0 msgr2=0x7f7cdc19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc19cfa0 0x7f7cdc19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 -- 172.21.15.37:0/166483749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc19da90 con 0x7f7cdc106170 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 0x7f7cdc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc19cfa0 0x7f7cdc19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc198c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000e9c0 tx=0x7f7cd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7cd8ff9640 1 -- 172.21.15.37:0/166483749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd000cd80 con 0x7f7cdc106170 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.090+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc19dd80 con 0x7f7cdc106170 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.091+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1a55c0 con 0x7f7cdc106170 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.091+0000 7f7cd8ff9640 1 -- 172.21.15.37:0/166483749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd00054d0 con 0x7f7cdc106170 2024-04-23T18:23:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.091+0000 7f7cd8ff9640 1 -- 172.21.15.37:0/166483749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0004e60 con 0x7f7cdc106170 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.093+0000 7f7cd8ff9640 1 -- 172.21.15.37:0/166483749 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7cd00046c0 con 0x7f7cdc106170 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 Processor -- start 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 -- start start 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cd8ff9640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7cc40a3d20 0x7f7cc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 0x557dde6f5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde6ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde700090 0x557dde702480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557dde65a370 con 0x557dde700090 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557dde5f72e0 con 0x557dde6f50a0 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557dde656330 con 0x557dde6f59c0 2024-04-23T18:23:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cd8ff9640 1 -- 172.21.15.37:0/166483749 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7cd00d0a70 con 0x7f7cdc106170 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7cc40a3d20 0x7f7cc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 0x557dde6f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 0x557dde6f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:53136/0 (socket says 172.21.15.37:53136) 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2142766389 learned_addr learned my addr 172.21.15.37:0/2142766389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde6ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde700090 0x557dde702480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2142766389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 msgr2=0x557dde6ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde6ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2142766389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde700090 msgr2=0x557dde702480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.094+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde700090 0x557dde702480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.095+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2142766389 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dde438d80 con 0x557dde6f50a0 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.095+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7cc40a3d20 0x7f7cc40a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4006010 tx=0x7f7cd4033000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.095+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 0x557dde6f5480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0115c0 tx=0x7f7ccc011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2dcf07842eef796 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.095+0000 7f7cc1ffb640 1 -- 172.21.15.37:0/2142766389 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00f640 con 0x557dde6f50a0 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.095+0000 7f7cc1ffb640 1 -- 172.21.15.37:0/2142766389 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc00fc90 con 0x557dde6f50a0 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.095+0000 7f7cc1ffb640 1 -- 172.21.15.37:0/2142766389 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc019c50 con 0x557dde6f50a0 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/2142766389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 msgr2=0x557dde6f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 0x557dde6f5480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0115c0 tx=0x7f7ccc011a80 comp rx=0 tx=0).stop 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/2142766389 shutdown_connections 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde700090 0x557dde702480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde6ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/2142766389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f50a0 0x557dde6f5480 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/2142766389 >> 172.21.15.37:0/2142766389 conn(0x557dde6f13c0 msgr2=0x557dde6f17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/2142766389 shutdown_connections 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/2142766389 wait complete. 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.096+0000 7f7ce4c1b740 1 -- start start 2024-04-23T18:23:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 Processor -- start 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f59c0 0x557dde6fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde700090 0x557dde6fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557dde702e80 con 0x557dde6f50a0 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557dde5f4fb0 con 0x557dde6f59c0 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557dde703000 con 0x557dde700090 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7ce4c1b740 10 client.0 _ll_register_callbacks cb 0x557dde5f49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59802/0 (socket says 172.21.15.37:59802) 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde700090 0x557dde6fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/3086737943 learned_addr learned my addr 172.21.15.37:0/3086737943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f59c0 0x557dde6fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/3086737943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde700090 msgr2=0x557dde6fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde700090 0x557dde6fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f59c0 msgr2=0x557dde6fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f59c0 0x557dde6fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/3086737943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dde499510 con 0x557dde6f50a0 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde700090 0x557dde6fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.097+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f59c0 0x557dde6fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6fcf80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0197f0 tx=0x7f7ccc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85eee8b5782679e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cbaffd640 1 -- 172.21.15.37:0/3086737943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc026020 con 0x557dde6f50a0 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cbaffd640 1 -- 172.21.15.37:0/3086737943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7ccc00f850 con 0x557dde6f50a0 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ccc00fc90 con 0x557dde6f50a0 2024-04-23T18:23:45.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cbaffd640 1 -- 172.21.15.37:0/3086737943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ccc031050 con 0x557dde6f50a0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7ce4c1b740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dde32c670 con 0x557dde6f50a0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cbaffd640 1 -- 172.21.15.37:0/3086737943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc00fc90 con 0x557dde6f50a0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.098+0000 7f7cbaffd640 1 -- 172.21.15.37:0/3086737943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7ccc0035f0 con 0x557dde6f50a0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7ce4c1b740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7ce4c1b740 10 client.15894 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7ce4c1b740 20 client.15894 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7103d0 0x557dde7127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7ce4c1b740 4 client.15894 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dde667d90 con 0x557dde7103d0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7103d0 0x557dde7127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.099+0000 7f7cbaffd640 10 client.15894 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7103d0 0x557dde7127c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70001046e85ac4a7 server_cookie=40922fbdf8b10e1b in_seq=0 out_seq=0 2024-04-23T18:23:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7cbaffd640 1 -- 172.21.15.37:0/3086737943 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dde667d90 con 0x557dde7103d0 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7cbaffd640 10 client.15894 handle_command_reply: tid=0 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 client.15894 shutdown 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 msgr2=0x557dde6fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6fcf80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0197f0 tx=0x7f7ccc0042f0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 shutdown_connections 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7103d0 0x557dde7127c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde700090 0x557dde6fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde6f59c0 0x557dde6fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7cbaffd640 0 client.15894 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/3086737943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6fcf80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.100+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 >> 172.21.15.37:0/3086737943 conn(0x557dde6f13c0 msgr2=0x557dde6f70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.101+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 shutdown_connections 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.101+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/3086737943 wait complete. 2024-04-23T18:23:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.101+0000 7f7ce4c1b740 20 client.15894 trim_cache size 0 max 16384 2024-04-23T18:23:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.119+0000 7f7ce4c1b740 1 Processor -- start 2024-04-23T18:23:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 -- start start 2024-04-23T18:23:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde718070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde7185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8011120 con 0x557dde6f50a0 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc800e990 con 0x557dde700090 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc800c7e0 con 0x557dde6f59c0 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde718070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59810/0 (socket says 172.21.15.37:59810) 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/1327293668 learned_addr learned my addr 172.21.15.37:0/1327293668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde7185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 -- 172.21.15.37:0/1327293668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 msgr2=0x557dde7185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde7185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 -- 172.21.15.37:0/1327293668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 msgr2=0x557dde6f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 -- 172.21.15.37:0/1327293668 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dde6f57f0 con 0x557dde6f59c0 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.120+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde718070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00ce1b0 tx=0x7f7cd009c5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=328b9f178ec5ba70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7cc17fa640 1 -- 172.21.15.37:0/1327293668 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd009ca00 con 0x557dde6f59c0 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7cc17fa640 1 -- 172.21.15.37:0/1327293668 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd00a0db0 con 0x557dde6f59c0 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7cc17fa640 1 -- 172.21.15.37:0/1327293668 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0087210 con 0x557dde6f59c0 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/1327293668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 msgr2=0x557dde718070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde718070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00ce1b0 tx=0x7f7cd009c5e0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/1327293668 shutdown_connections 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde7185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde718070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/1327293668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde6f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/1327293668 >> 172.21.15.37:0/1327293668 conn(0x557dde6f13c0 msgr2=0x557dde6fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.121+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/1327293668 shutdown_connections 2024-04-23T18:23:45.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/1327293668 wait complete. 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 -- start start 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 Processor -- start 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde703ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde704430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde704970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc800ce50 con 0x557dde6f50a0 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc800e2c0 con 0x557dde700090 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc800d520 con 0x557dde6f59c0 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7ce4c1b740 10 client.0 _ll_register_callbacks cb 0x7f7ccc011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde703ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde703ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59818/0 (socket says 172.21.15.37:59818) 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/264002043 learned_addr learned my addr 172.21.15.37:0/264002043 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde704970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.122+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde704430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/264002043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 msgr2=0x557dde704430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde704430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 msgr2=0x557dde704970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde704970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/264002043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dde775b30 con 0x557dde6f50a0 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdbfff640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde704970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde703ef0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc003bb0 tx=0x7f7ccc00f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4b1fa65eeb215e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cbaffd640 1 -- 172.21.15.37:0/264002043 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc026040 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cbaffd640 1 -- 172.21.15.37:0/264002043 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7ccc0055d0 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd00a0db0 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7cbaffd640 1 -- 172.21.15.37:0/264002043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ccc003df0 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7ce4c1b740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.123+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dde776dd0 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7cbaffd640 1 -- 172.21.15.37:0/264002043 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd00a0db0 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7cbaffd640 1 -- 172.21.15.37:0/264002043 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7ccc019ac0 con 0x557dde6f50a0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7ce4c1b740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7ce4c1b740 10 client.15900 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7ce4c1b740 20 client.15900 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7775b0 0x557dde7799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7ce4c1b740 4 client.15900 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.35, "await_for": 42.0}] 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.35, "await_for": 42.0}) -- 0x557dde661c00 con 0x557dde7775b0 2024-04-23T18:23:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.124+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7775b0 0x557dde7799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.125+0000 7f7cbaffd640 10 client.15900 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.125+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7775b0 0x557dde7799d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163367e0b9048277 server_cookie=56adef530fc00680 in_seq=0 out_seq=0 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7cbaffd640 1 -- 172.21.15.37:0/264002043 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557dde661c00 con 0x557dde7775b0 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7cbaffd640 10 client.15900 handle_command_reply: tid=0 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 client.15900 shutdown 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 msgr2=0x557dde703ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde703ef0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc003bb0 tx=0x7f7ccc00f430 comp rx=0 tx=0).stop 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 shutdown_connections 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557dde7775b0 0x557dde7799d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557dde700090 0x557dde704970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557dde6f59c0 0x557dde704430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7cbaffd640 0 client.15900 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 --2- 172.21.15.37:0/264002043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557dde6f50a0 0x557dde703ef0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 >> 172.21.15.37:0/264002043 conn(0x557dde6f13c0 msgr2=0x557dde6fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 shutdown_connections 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.185+0000 7f7ce4c1b740 1 -- 172.21.15.37:0/264002043 wait complete. 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce4c1b740 20 client.15900 trim_cache size 0 max 16384 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 28, 2024-04-23T18:23:45.180 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 10, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "2990d434": { 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 10, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 80.3, 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:23:45.181 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:23:45.182 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7cc40a3d20 msgr2=0x7f7cc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7cc40a3d20 0x7f7cc40a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4006010 tx=0x7f7cd4033000 comp rx=0 tx=0).stop 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 msgr2=0x7f7cdc198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc198c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000e9c0 tx=0x7f7cd000ee80 comp rx=0 tx=0).stop 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 shutdown_connections 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7cc40a3d20 0x7f7cc40a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cdc19cfa0 0x7f7cdc19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc106170 0x7f7cdc198c20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 --2- 172.21.15.37:0/166483749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cdc1010a0 0x7f7cdc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.186+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 >> 172.21.15.37:0/166483749 conn(0x7f7cdc077f50 msgr2=0x7f7cdc0693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.187+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 shutdown_connections 2024-04-23T18:23:45.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:23:45.187+0000 7f7ce1e2a640 1 -- 172.21.15.37:0/166483749 wait complete. 2024-04-23T18:23:45.191 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2990d434', quiesce took 0.7 seconds. Will release after: 20.35 2024-04-23T18:23:45.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:45 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/211508091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:45.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:45 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/211508091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:45.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:45 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/211508091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:45.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.90 MB/sec warmup 110 sec latency 676.146 ms 2024-04-23T18:23:46.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:46 smithi172 ceph-mon[32456]: pgmap v102: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-04-23T18:23:46.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:46 smithi173 ceph-mon[32906]: pgmap v102: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-04-23T18:23:46.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:46 smithi037 ceph-mon[29404]: pgmap v102: 97 pgs: 97 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-04-23T18:23:46.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.80 MB/sec warmup 111 sec latency 1676.265 ms 2024-04-23T18:23:47.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.69 MB/sec warmup 112 sec latency 2676.320 ms 2024-04-23T18:23:48.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:48 smithi172 ceph-mon[32456]: pgmap v103: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 364 op/s 2024-04-23T18:23:48.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:48 smithi173 ceph-mon[32906]: pgmap v103: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 364 op/s 2024-04-23T18:23:48.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:48 smithi037 ceph-mon[29404]: pgmap v103: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 364 op/s 2024-04-23T18:23:48.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.59 MB/sec warmup 113 sec latency 3676.417 ms 2024-04-23T18:23:49.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.49 MB/sec warmup 114 sec latency 4676.535 ms 2024-04-23T18:23:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/2898171549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b98102b70 msgr2=0x7f2b98106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/2898171549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b98102b70 0x7f2b98106bb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009900 tx=0x7f2b8c02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/2898171549 shutdown_connections 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/2898171549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b98107290 0x7f2b9810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/2898171549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b98102b70 0x7f2b98106bb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/2898171549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b981021c0 0x7f2b981025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/2898171549 >> 172.21.15.37:0/2898171549 conn(0x7f2b980fda20 msgr2=0x7f2b980ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/2898171549 shutdown_connections 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.154+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/2898171549 wait complete. 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 Processor -- start 2024-04-23T18:23:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 -- start start 2024-04-23T18:23:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 0x7f2b981986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b98107290 0x7f2b98198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b9819cfb0 0x7f2b9819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981121d0 con 0x7f2b981021c0 2024-04-23T18:23:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98112050 con 0x7f2b98107290 2024-04-23T18:23:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b9e9ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98112350 con 0x7f2b9819cfb0 2024-04-23T18:23:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 0x7f2b981986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 0x7f2b981986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59834/0 (socket says 172.21.15.37:59834) 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b97fff640 1 -- 172.21.15.37:0/1700393801 learned_addr learned my addr 172.21.15.37:0/1700393801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.155+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b98107290 0x7f2b98198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b9819cfb0 0x7f2b9819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b97fff640 1 -- 172.21.15.37:0/1700393801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b9819cfb0 msgr2=0x7f2b9819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b97fff640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b9819cfb0 0x7f2b9819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b97fff640 1 -- 172.21.15.37:0/1700393801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b98107290 msgr2=0x7f2b98198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b97fff640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b98107290 0x7f2b98198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b97fff640 1 -- 172.21.15.37:0/1700393801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9819daa0 con 0x7f2b981021c0 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b97fff640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 0x7f2b981986f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800e9f0 tx=0x7f2b8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b957fa640 1 -- 172.21.15.37:0/1700393801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800cd80 con 0x7f2b981021c0 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b957fa640 1 -- 172.21.15.37:0/1700393801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b880054d0 con 0x7f2b981021c0 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9819dd90 con 0x7f2b981021c0 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b957fa640 1 -- 172.21.15.37:0/1700393801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88010690 con 0x7f2b981021c0 2024-04-23T18:23:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.156+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b981a5500 con 0x7f2b981021c0 2024-04-23T18:23:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2b957fa640 1 -- 172.21.15.37:0/1700393801 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b88004e90 con 0x7f2b981021c0 2024-04-23T18:23:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 Processor -- start 2024-04-23T18:23:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 -- start start 2024-04-23T18:23:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 0x557db21ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2b957fa640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2b800a3dc0 0x7f2b800a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21bae40 0x557db21c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21c9c10 0x557db21cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557db211f970 con 0x557db21ba520 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557db20bc4a0 con 0x557db21c9c10 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557db211b930 con 0x557db21bae40 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2b97fff640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2b800a3dc0 0x7f2b800a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.159+0000 7f2b9cf4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 0x557db21ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 0x557db21ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59850/0 (socket says 172.21.15.37:59850) 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b977fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21bae40 0x557db21c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/2910680491 learned_addr learned my addr 172.21.15.37:0/2910680491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b957fa640 1 -- 172.21.15.37:0/1700393801 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2b880d0c10 con 0x7f2b981021c0 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21c9c10 0x557db21cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/2910680491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21bae40 msgr2=0x557db21c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21bae40 0x557db21c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/2910680491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21c9c10 msgr2=0x557db21cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21c9c10 0x557db21cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/2910680491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db1e32210 con 0x557db21ba520 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b97fff640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2b800a3dc0 0x7f2b800a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c6a0 tx=0x7f2b8806e6a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 0x557db21ba900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005c40 tx=0x7f2b90009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc630061bdd362d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b7e7fc640 1 -- 172.21.15.37:0/2910680491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000e850 con 0x557db21ba520 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b7e7fc640 1 -- 172.21.15.37:0/2910680491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b90010460 con 0x557db21ba520 2024-04-23T18:23:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.160+0000 7f2b7e7fc640 1 -- 172.21.15.37:0/2910680491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000eea0 con 0x557db21ba520 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 -- 172.21.15.37:0/2910680491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 msgr2=0x557db21ba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 0x557db21ba900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005c40 tx=0x7f2b90009b30 comp rx=0 tx=0).stop 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 -- 172.21.15.37:0/2910680491 shutdown_connections 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21c9c10 0x557db21cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21bae40 0x557db21c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/2910680491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21ba520 0x557db21ba900 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 -- 172.21.15.37:0/2910680491 >> 172.21.15.37:0/2910680491 conn(0x557db21b6b30 msgr2=0x557db21b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 -- 172.21.15.37:0/2910680491 shutdown_connections 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 -- 172.21.15.37:0/2910680491 wait complete. 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.161+0000 7f2ba17a2740 1 -- start start 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 Processor -- start 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557db21cc840 con 0x557db21c9c10 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557db20ba0b0 con 0x557db21bae40 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557db21cc9c0 con 0x557db21ba520 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2ba17a2740 10 client.0 _ll_register_callbacks cb 0x557db20a7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59856/0 (socket says 172.21.15.37:59856) 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1450025156 learned_addr learned my addr 172.21.15.37:0/1450025156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1450025156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 msgr2=0x557db21c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 msgr2=0x557db21c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.162+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1450025156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db1efe620 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21c6cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005c10 tx=0x7f2b90010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861e3fff3f8051d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b777fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b777fe640 1 -- 172.21.15.37:0/1450025156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90004c50 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b777fe640 1 -- 172.21.15.37:0/1450025156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2b90004df0 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b90010460 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b777fe640 1 -- 172.21.15.37:0/1450025156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b90003970 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2ba17a2740 10 client.15912 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557db21d8700 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b777fe640 1 -- 172.21.15.37:0/1450025156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b90010460 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2b777fe640 1 -- 172.21.15.37:0/1450025156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f2b9001d070 con 0x557db21c9c10 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2ba17a2740 10 client.15912 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2ba17a2740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.163+0000 7f2ba17a2740 20 client.15912 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.164+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db21d9a50 0x557db21dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.164+0000 7f2ba17a2740 4 client.15912 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.164+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557db212d390 con 0x557db21d9a50 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.164+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db21d9a50 0x557db21dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.164+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db21d9a50 0x557db21dbe40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29cda286cf9c7c8e server_cookie=cd7ed03f4b3c1422 in_seq=0 out_seq=0 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.164+0000 7f2b777fe640 10 client.15912 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2b777fe640 1 -- 172.21.15.37:0/1450025156 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557db212d390 con 0x557db21d9a50 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2b777fe640 10 client.15912 handle_command_reply: tid=0 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 client.15912 shutdown 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 msgr2=0x557db21c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21c6cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005c10 tx=0x7f2b90010d80 comp rx=0 tx=0).stop 2024-04-23T18:23:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 shutdown_connections 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db21d9a50 0x557db21dbe40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21c6cd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2b777fe640 0 client.15912 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1450025156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 >> 172.21.15.37:0/1450025156 conn(0x557db21b6b30 msgr2=0x557db21bc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 shutdown_connections 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1450025156 wait complete. 2024-04-23T18:23:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.165+0000 7f2ba17a2740 20 client.15912 trim_cache size 0 max 16384 2024-04-23T18:23:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.180+0000 7f2ba17a2740 1 Processor -- start 2024-04-23T18:23:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.180+0000 7f2ba17a2740 1 -- start start 2024-04-23T18:23:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.180+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21ba520 0x557db21c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 0x557db21c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21c9c10 0x557db21c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ae340 con 0x557db21bae40 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981abbb0 con 0x557db21ba520 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981a9a00 con 0x557db21c9c10 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 0x557db21c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 0x557db21c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59864/0 (socket says 172.21.15.37:59864) 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 -- 172.21.15.37:0/738017731 learned_addr learned my addr 172.21.15.37:0/738017731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21c9c10 0x557db21c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 -- 172.21.15.37:0/738017731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21c9c10 msgr2=0x557db21c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21c9c10 0x557db21c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 -- 172.21.15.37:0/738017731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21ba520 msgr2=0x557db21c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21ba520 0x557db21c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 -- 172.21.15.37:0/738017731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db21bb380 con 0x557db21bae40 2024-04-23T18:23:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21c9c10 0x557db21c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b977fe640 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 0x557db21c8fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000c00 tx=0x7f2b8c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef629579a500294e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b7dffb640 1 -- 172.21.15.37:0/738017731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c04b070 con 0x557db21bae40 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b7dffb640 1 -- 172.21.15.37:0/738017731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b8c046dc0 con 0x557db21bae40 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.181+0000 7f2b7dffb640 1 -- 172.21.15.37:0/738017731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c035040 con 0x557db21bae40 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 -- 172.21.15.37:0/738017731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 msgr2=0x557db21c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 0x557db21c8fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000c00 tx=0x7f2b8c031400 comp rx=0 tx=0).stop 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 -- 172.21.15.37:0/738017731 shutdown_connections 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21c9c10 0x557db21c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21bae40 0x557db21c8fd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/738017731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21ba520 0x557db21c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 -- 172.21.15.37:0/738017731 >> 172.21.15.37:0/738017731 conn(0x557db21b6b30 msgr2=0x557db21b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 -- 172.21.15.37:0/738017731 shutdown_connections 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 -- 172.21.15.37:0/738017731 wait complete. 2024-04-23T18:23:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 -- start start 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.182+0000 7f2ba17a2740 1 Processor -- start 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21be290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981aa070 con 0x557db21c9c10 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ab4e0 con 0x557db21bae40 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981aa740 con 0x557db21ba520 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2ba17a2740 10 client.0 _ll_register_callbacks cb 0x7f2b900123d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21be7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59880/0 (socket says 172.21.15.37:59880) 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1947105647 learned_addr learned my addr 172.21.15.37:0/1947105647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1947105647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 msgr2=0x557db21d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 msgr2=0x557db21be290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21be290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 -- 172.21.15.37:0/1947105647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db21b6960 con 0x557db21c9c10 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21be290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b9cf4a640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21be7d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009ac0 tx=0x7f2b900097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c25d81a00872aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.183+0000 7f2b777fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2b777fe640 1 -- 172.21.15.37:0/1947105647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b900108a0 con 0x557db21c9c10 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2b777fe640 1 -- 172.21.15.37:0/1947105647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2b90010a40 con 0x557db21c9c10 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b8c046dc0 con 0x557db21c9c10 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2b777fe640 1 -- 172.21.15.37:0/1947105647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b90010c40 con 0x557db21c9c10 2024-04-23T18:23:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557db21c72d0 con 0x557db21c9c10 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2b777fe640 1 -- 172.21.15.37:0/1947105647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b8c046dc0 con 0x557db21c9c10 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2b777fe640 1 -- 172.21.15.37:0/1947105647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f2b9001d070 con 0x557db21c9c10 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 20 client.15918 populate_metadata read hostname 'smithi037' 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db22447e0 0x557db2246bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 4 client.15918 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.184+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557db21c7060 con 0x557db22447e0 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.185+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db22447e0 0x557db2246bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.185+0000 7f2b777fe640 10 client.15918 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.185+0000 7f2b977fe640 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db22447e0 0x557db2246bd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c60dd89c567006 server_cookie=d0aa0d0ad5d7ccb5 in_seq=0 out_seq=0 2024-04-23T18:23:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.234+0000 7f2b777fe640 1 -- 172.21.15.37:0/1947105647 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557db21c7060 con 0x557db22447e0 2024-04-23T18:23:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.234+0000 7f2b777fe640 10 client.15918 handle_command_reply: tid=0 2024-04-23T18:23:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.234+0000 7f2ba17a2740 1 client.15918 shutdown 2024-04-23T18:23:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 msgr2=0x557db21be7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21be7d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009ac0 tx=0x7f2b900097d0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 shutdown_connections 2024-04-23T18:23:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557db22447e0 0x557db2246bd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557db21c9c10 0x557db21be7d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db21bae40 0x557db21be290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 --2- 172.21.15.37:0/1947105647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557db21ba520 0x557db21d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 >> 172.21.15.37:0/1947105647 conn(0x557db21b6b30 msgr2=0x557db21d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2b777fe640 0 client.15918 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 shutdown_connections 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 1 -- 172.21.15.37:0/1947105647 wait complete. 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.235+0000 7f2ba17a2740 20 client.15918 trim_cache size 0 max 16384 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:23:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.236+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2b800a3dc0 msgr2=0x7f2b800a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2b800a3dc0 0x7f2b800a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c6a0 tx=0x7f2b8806e6a0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 msgr2=0x7f2b981986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 0x7f2b981986f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800e9f0 tx=0x7f2b8800edf0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 shutdown_connections 2024-04-23T18:23:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2b800a3dc0 0x7f2b800a6280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b9819cfb0 0x7f2b9819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b98107290 0x7f2b98198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 --2- 172.21.15.37:0/1700393801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b981021c0 0x7f2b981986f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 >> 172.21.15.37:0/1700393801 conn(0x7f2b980fda20 msgr2=0x7f2b980ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 shutdown_connections 2024-04-23T18:23:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:23:50.237+0000 7f2b9e9ad640 1 -- 172.21.15.37:0/1700393801 wait complete. 2024-04-23T18:23:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:50 smithi172 ceph-mon[32456]: pgmap v104: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-23T18:23:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:50 smithi173 ceph-mon[32906]: pgmap v104: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-23T18:23:50.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:50 smithi037 ceph-mon[29404]: pgmap v104: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-23T18:23:50.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.39 MB/sec warmup 115 sec latency 5676.626 ms 2024-04-23T18:23:51.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.29 MB/sec warmup 116 sec latency 6676.731 ms 2024-04-23T18:23:52.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:52 smithi172 ceph-mon[32456]: pgmap v105: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-23T18:23:52.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:52 smithi173 ceph-mon[32906]: pgmap v105: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-23T18:23:52.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:52 smithi037 ceph-mon[29404]: pgmap v105: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-23T18:23:52.812 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.19 MB/sec warmup 117 sec latency 7676.848 ms 2024-04-23T18:23:53.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.10 MB/sec warmup 118 sec latency 8676.908 ms 2024-04-23T18:23:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:54 smithi172 ceph-mon[32456]: pgmap v106: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-23T18:23:54.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:54 smithi173 ceph-mon[32906]: pgmap v106: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-23T18:23:54.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:54 smithi037 ceph-mon[29404]: pgmap v106: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-23T18:23:54.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 11.00 MB/sec warmup 119 sec latency 9677.020 ms 2024-04-23T18:23:56.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:56 smithi172 ceph-mon[32456]: pgmap v107: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.9 MiB/s wr, 151 op/s 2024-04-23T18:23:56.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:56 smithi173 ceph-mon[32906]: pgmap v107: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.9 MiB/s wr, 151 op/s 2024-04-23T18:23:56.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:56 smithi037 ceph-mon[29404]: pgmap v107: 97 pgs: 97 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.9 MiB/s wr, 151 op/s 2024-04-23T18:23:56.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 1 sec latency 11677.218 ms 2024-04-23T18:23:57.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 2 sec latency 12677.303 ms 2024-04-23T18:23:58.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:23:58 smithi172 ceph-mon[32456]: pgmap v108: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 170 op/s 2024-04-23T18:23:58.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:23:58 smithi173 ceph-mon[32906]: pgmap v108: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 170 op/s 2024-04-23T18:23:58.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:23:58 smithi037 ceph-mon[29404]: pgmap v108: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 170 op/s 2024-04-23T18:23:58.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 3 sec latency 13677.407 ms 2024-04-23T18:23:59.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 4 sec latency 14677.507 ms 2024-04-23T18:24:00.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:00 smithi172 ceph-mon[32456]: pgmap v109: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-04-23T18:24:00.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:00 smithi173 ceph-mon[32906]: pgmap v109: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-04-23T18:24:00.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:00 smithi037 ceph-mon[29404]: pgmap v109: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-04-23T18:24:00.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 5 sec latency 15677.617 ms 2024-04-23T18:24:01.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 6 sec latency 16677.733 ms 2024-04-23T18:24:02.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:02 smithi172 ceph-mon[32456]: pgmap v110: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-04-23T18:24:02.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:02 smithi173 ceph-mon[32906]: pgmap v110: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-04-23T18:24:02.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:02 smithi037 ceph-mon[29404]: pgmap v110: 97 pgs: 97 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.4 MiB/s wr, 68 op/s 2024-04-23T18:24:02.813 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 7 sec latency 17677.857 ms 2024-04-23T18:24:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 -- 172.21.15.37:0/1995871433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea60101a90 msgr2=0x7fea60105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 --2- 172.21.15.37:0/1995871433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea60101a90 0x7fea60105ad0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fea50009900 tx=0x7fea5002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 -- 172.21.15.37:0/1995871433 shutdown_connections 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 --2- 172.21.15.37:0/1995871433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea601061b0 0x7fea6010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 --2- 172.21.15.37:0/1995871433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea60101a90 0x7fea60105ad0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 --2- 172.21.15.37:0/1995871433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601010e0 0x7fea601014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 -- 172.21.15.37:0/1995871433 >> 172.21.15.37:0/1995871433 conn(0x7fea600771d0 msgr2=0x7fea600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 -- 172.21.15.37:0/1995871433 shutdown_connections 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.338+0000 7fea672dc640 1 -- 172.21.15.37:0/1995871433 wait complete. 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 Processor -- start 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 -- start start 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 0x7fea60198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601061b0 0x7fea60198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea6019cf00 0x7fea6019d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea601110f0 con 0x7fea601010e0 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea60110f70 con 0x7fea601061b0 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea672dc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea60111270 con 0x7fea6019cf00 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 0x7fea60198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 0x7fea60198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52300/0 (socket says 172.21.15.37:52300) 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 -- 172.21.15.37:0/1317165896 learned_addr learned my addr 172.21.15.37:0/1317165896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65879640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea6019cf00 0x7fea6019d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea64877640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601061b0 0x7fea60198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 -- 172.21.15.37:0/1317165896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea6019cf00 msgr2=0x7fea6019d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea6019cf00 0x7fea6019d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 -- 172.21.15.37:0/1317165896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601061b0 msgr2=0x7fea60198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601061b0 0x7fea60198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea65078640 1 -- 172.21.15.37:0/1317165896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6019d9f0 con 0x7fea601010e0 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.339+0000 7fea64877640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601061b0 0x7fea60198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.340+0000 7fea65078640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 0x7fea60198640 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00d950 tx=0x7fea5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.340+0000 7fea567fc640 1 -- 172.21.15.37:0/1317165896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c017030 con 0x7fea601010e0 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.340+0000 7fea567fc640 1 -- 172.21.15.37:0/1317165896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c004020 con 0x7fea601010e0 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.340+0000 7fea567fc640 1 -- 172.21.15.37:0/1317165896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0047a0 con 0x7fea601010e0 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.340+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6019dce0 con 0x7fea601010e0 2024-04-23T18:24:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.340+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea601a54f0 con 0x7fea601010e0 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.342+0000 7fea567fc640 1 -- 172.21.15.37:0/1317165896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea5c009a30 con 0x7fea601010e0 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 Processor -- start 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 -- start start 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c57732a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563c57697970 con 0x563c5773d800 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563c576344a0 con 0x563c57732620 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea69fea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563c57693930 con 0x563c57732f40 2024-04-23T18:24:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea65078640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52314/0 (socket says 172.21.15.37:52314) 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 -- 172.21.15.37:0/627259373 learned_addr learned my addr 172.21.15.37:0/627259373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea65879640 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c57732a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea567fc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fea4c0a3e40 0x7fea4c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea65879640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fea4c0a3e40 0x7fea4c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 -- 172.21.15.37:0/627259373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 msgr2=0x563c5773d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 -- 172.21.15.37:0/627259373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 msgr2=0x563c57732a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c57732a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea64877640 1 -- 172.21.15.37:0/627259373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c573aa210 con 0x563c5773d800 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.343+0000 7fea567fc640 1 -- 172.21.15.37:0/1317165896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fea5c0d0200 con 0x7fea601010e0 2024-04-23T18:24:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.344+0000 7fea64877640 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773fbf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fea6006a570 tx=0x7fea50032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9b2a40f77cda1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.344+0000 7fea65879640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fea4c0a3e40 0x7fea4c0a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea58009fd0 tx=0x7fea58004ab0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.344+0000 7fea4b7fe640 1 -- 172.21.15.37:0/627259373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5003f070 con 0x563c5773d800 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.344+0000 7fea4b7fe640 1 -- 172.21.15.37:0/627259373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea50032e10 con 0x563c5773d800 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.344+0000 7fea4b7fe640 1 -- 172.21.15.37:0/627259373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50035070 con 0x563c5773d800 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 -- 172.21.15.37:0/627259373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 msgr2=0x563c5773fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773fbf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fea6006a570 tx=0x7fea50032c20 comp rx=0 tx=0).stop 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 -- 172.21.15.37:0/627259373 shutdown_connections 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773fbf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 --2- 172.21.15.37:0/627259373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c57732a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 -- 172.21.15.37:0/627259373 >> 172.21.15.37:0/627259373 conn(0x563c5772eb30 msgr2=0x563c5772ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 -- 172.21.15.37:0/627259373 shutdown_connections 2024-04-23T18:24:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 -- 172.21.15.37:0/627259373 wait complete. 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 -- start start 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.345+0000 7fea69fea740 1 Processor -- start 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c5773d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563c577403d0 con 0x563c5773d800 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563c576320b0 con 0x563c57732620 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563c57740550 con 0x563c57732f40 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea69fea740 10 client.0 _ll_register_callbacks cb 0x563c5761fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52326/0 (socket says 172.21.15.37:52326) 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 -- 172.21.15.37:0/1793302668 learned_addr learned my addr 172.21.15.37:0/1793302668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65078640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c5773d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea64877640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 -- 172.21.15.37:0/1793302668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 msgr2=0x563c5773a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 -- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 msgr2=0x563c5773d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c5773d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 -- 172.21.15.37:0/1793302668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c57476620 con 0x563c5773d800 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea65879640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773a900 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fea58012040 tx=0x7fea58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6240f543a44b23b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.346+0000 7fea48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea48ff9640 1 -- 172.21.15.37:0/1793302668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58025070 con 0x563c5773d800 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea48ff9640 1 -- 172.21.15.37:0/1793302668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fea58010dc0 con 0x563c5773d800 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50032e10 con 0x563c5773d800 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea48ff9640 1 -- 172.21.15.37:0/1793302668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea5802a460 con 0x563c5773d800 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 10 client.15930 fetch_fsmap learned FSMap version 28 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c5774c2d0 con 0x563c5773d800 2024-04-23T18:24:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea48ff9640 1 -- 172.21.15.37:0/1793302668 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea50032e10 con 0x563c5773d800 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea48ff9640 1 -- 172.21.15.37:0/1793302668 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fea58021560 con 0x563c5773d800 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 20 client.15930 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c5774d740 0x563c5774fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 4 client.15930 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.347+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c576a5390 con 0x563c5774d740 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.348+0000 7fea65078640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c5774d740 0x563c5774fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.348+0000 7fea48ff9640 10 client.15930 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.348+0000 7fea65078640 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c5774d740 0x563c5774fb50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6245149e41867cb7 server_cookie=f4ec7549cc6292c2 in_seq=0 out_seq=0 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea48ff9640 1 -- 172.21.15.37:0/1793302668 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c576a5390 con 0x563c5774d740 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea48ff9640 10 client.15930 handle_command_reply: tid=0 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 client.15930 shutdown 2024-04-23T18:24:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 msgr2=0x563c5773a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773a900 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fea58012040 tx=0x7fea58004a40 comp rx=0 tx=0).stop 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 shutdown_connections 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c5774d740 0x563c5774fb50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c5773d800 0x563c5773a900 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732f40 0x563c5773a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea48ff9640 0 client.15930 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 --2- 172.21.15.37:0/1793302668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c57732620 0x563c5773d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 >> 172.21.15.37:0/1793302668 conn(0x563c5772eb30 msgr2=0x563c57734860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 shutdown_connections 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 1 -- 172.21.15.37:0/1793302668 wait complete. 2024-04-23T18:24:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.349+0000 7fea69fea740 20 client.15930 trim_cache size 0 max 16384 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 Processor -- start 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 -- start start 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57749290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57755460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c577559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea601b03f0 con 0x563c57732f40 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea601adc60 con 0x563c5773d800 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea69fea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea601abab0 con 0x563c57732620 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea64877640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57755460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea64877640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57755460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52342/0 (socket says 172.21.15.37:52342) 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.365+0000 7fea64877640 1 -- 172.21.15.37:0/1170081851 learned_addr learned my addr 172.21.15.37:0/1170081851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea65078640 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57749290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea65879640 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c577559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea64877640 1 -- 172.21.15.37:0/1170081851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 msgr2=0x563c57749290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea64877640 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57749290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea64877640 1 -- 172.21.15.37:0/1170081851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 msgr2=0x563c577559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea64877640 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c577559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea64877640 1 -- 172.21.15.37:0/1170081851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5772f700 con 0x563c57732f40 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea64877640 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57755460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fea50033000 tx=0x7fea500051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759ddb5fc6e54aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea4affd640 1 -- 172.21.15.37:0/1170081851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5003f070 con 0x563c57732f40 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea4affd640 1 -- 172.21.15.37:0/1170081851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5004b070 con 0x563c57732f40 2024-04-23T18:24:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.366+0000 7fea4affd640 1 -- 172.21.15.37:0/1170081851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50035040 con 0x563c57732f40 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 -- 172.21.15.37:0/1170081851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 msgr2=0x563c57755460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57755460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fea50033000 tx=0x7fea500051e0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 -- 172.21.15.37:0/1170081851 shutdown_connections 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c577559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57755460 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 --2- 172.21.15.37:0/1170081851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57749290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 -- 172.21.15.37:0/1170081851 >> 172.21.15.37:0/1170081851 conn(0x563c5772eb30 msgr2=0x563c5772d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 -- 172.21.15.37:0/1170081851 shutdown_connections 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 -- 172.21.15.37:0/1170081851 wait complete. 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 -- start start 2024-04-23T18:24:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.367+0000 7fea69fea740 1 Processor -- start 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57743100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57743640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c57743b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ac120 con 0x563c57732f40 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ad590 con 0x563c5773d800 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ac7f0 con 0x563c57732620 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea69fea740 10 client.0 _ll_register_callbacks cb 0x7fea58013240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57743640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57743640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52348/0 (socket says 172.21.15.37:52348) 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 -- 172.21.15.37:0/470230131 learned_addr learned my addr 172.21.15.37:0/470230131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea65879640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c57743b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea65078640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57743100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 -- 172.21.15.37:0/470230131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 msgr2=0x563c57743100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57743100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 -- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 msgr2=0x563c57743b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c57743b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea64877640 1 -- 172.21.15.37:0/470230131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c577b2ce0 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.368+0000 7fea65078640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57743100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.369+0000 7fea48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.369+0000 7fea64877640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57743640 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea50006010 tx=0x7fea50005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54b2733bccc6cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea48ff9640 1 -- 172.21.15.37:0/470230131 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50053020 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea48ff9640 1 -- 172.21.15.37:0/470230131 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fea500035f0 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5004b070 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea48ff9640 1 -- 172.21.15.37:0/470230131 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea5005e050 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea69fea740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c577b3f00 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.370+0000 7fea48ff9640 1 -- 172.21.15.37:0/470230131 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5004b070 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea48ff9640 1 -- 172.21.15.37:0/470230131 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fea50004990 con 0x563c57732f40 2024-04-23T18:24:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea69fea740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea69fea740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea69fea740 20 client.15936 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea69fea740 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c577b41a0 0x563c577b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea69fea740 4 client.15936 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c5769f200 con 0x563c577b41a0 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.371+0000 7fea65078640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c577b41a0 0x563c577b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.372+0000 7fea48ff9640 10 client.15936 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.372+0000 7fea65078640 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c577b41a0 0x563c577b65c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f87ac87ea9cb60 server_cookie=6026272a51137b02 in_seq=0 out_seq=0 2024-04-23T18:24:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea48ff9640 1 -- 172.21.15.37:0/470230131 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c5769f200 con 0x563c577b41a0 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea48ff9640 10 client.15936 handle_command_reply: tid=0 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 client.15936 shutdown 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 msgr2=0x563c57743640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57743640 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea50006010 tx=0x7fea50005350 comp rx=0 tx=0).stop 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 shutdown_connections 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563c577b41a0 0x563c577b65c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563c5773d800 0x563c57743b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea48ff9640 0 client.15936 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c57732f40 0x563c57743640 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 --2- 172.21.15.37:0/470230131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563c57732620 0x563c57743100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 >> 172.21.15.37:0/470230131 conn(0x563c5772eb30 msgr2=0x563c5772dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 shutdown_connections 2024-04-23T18:24:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 1 -- 172.21.15.37:0/470230131 wait complete. 2024-04-23T18:24:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.553+0000 7fea69fea740 20 client.15936 trim_cache size 0 max 16384 2024-04-23T18:24:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:24:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:24:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:24:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fea4c0a3e40 msgr2=0x7fea4c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fea4c0a3e40 0x7fea4c0a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea58009fd0 tx=0x7fea58004ab0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 msgr2=0x7fea60198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 0x7fea60198640 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00d950 tx=0x7fea5c00de10 comp rx=0 tx=0).stop 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 shutdown_connections 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fea4c0a3e40 0x7fea4c0a6300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea6019cf00 0x7fea6019d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea601061b0 0x7fea60198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 --2- 172.21.15.37:0/1317165896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea601010e0 0x7fea60198640 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 >> 172.21.15.37:0/1317165896 conn(0x7fea600771d0 msgr2=0x7fea6006a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 shutdown_connections 2024-04-23T18:24:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:03.554+0000 7fea672dc640 1 -- 172.21.15.37:0/1317165896 wait complete. 2024-04-23T18:24:03.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 8 sec latency 18677.965 ms 2024-04-23T18:24:04.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:04 smithi172 ceph-mon[32456]: pgmap v111: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-04-23T18:24:04.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:04 smithi173 ceph-mon[32906]: pgmap v111: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-04-23T18:24:04.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:04 smithi037 ceph-mon[29404]: pgmap v111: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-04-23T18:24:04.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 9 sec latency 19678.092 ms 2024-04-23T18:24:05.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:05.567 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:05 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:05.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:05 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:05.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:05 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 -- 172.21.15.37:0/1586004343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb301021b0 msgr2=0x7feb30102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 --2- 172.21.15.37:0/1586004343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb301021b0 0x7feb30102590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feb20009ca0 tx=0x7feb2002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 -- 172.21.15.37:0/1586004343 shutdown_connections 2024-04-23T18:24:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 --2- 172.21.15.37:0/1586004343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 0x7feb3010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 --2- 172.21.15.37:0/1586004343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb30102b60 0x7feb30106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 --2- 172.21.15.37:0/1586004343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb301021b0 0x7feb30102590 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 -- 172.21.15.37:0/1586004343 >> 172.21.15.37:0/1586004343 conn(0x7feb300fd9f0 msgr2=0x7feb300ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 -- 172.21.15.37:0/1586004343 shutdown_connections 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 -- 172.21.15.37:0/1586004343 wait complete. 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.661+0000 7feb360dc640 1 Processor -- start 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 -- start start 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb301021b0 0x7feb30074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 0x7feb30074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 0x7feb301a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb301121c0 con 0x7feb30074cd0 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb30112040 con 0x7feb30107280 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb360dc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb30112340 con 0x7feb301021b0 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 0x7feb301a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 0x7feb301a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59118/0 (socket says 172.21.15.37:59118) 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 -- 172.21.15.37:0/1658647236 learned_addr learned my addr 172.21.15.37:0/1658647236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2f7fe640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb301021b0 0x7feb30074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2effd640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 0x7feb30074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 -- 172.21.15.37:0/1658647236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb301021b0 msgr2=0x7feb30074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb301021b0 0x7feb30074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 -- 172.21.15.37:0/1658647236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 msgr2=0x7feb30074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 0x7feb30074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 -- 172.21.15.37:0/1658647236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb301a7100 con 0x7feb30074cd0 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2effd640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 0x7feb30074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.662+0000 7feb2ffff640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 0x7feb301a6a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb2800d950 tx=0x7feb2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.663+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28017030 con 0x7feb30074cd0 2024-04-23T18:24:05.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.663+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb28004020 con 0x7feb30074cd0 2024-04-23T18:24:05.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.663+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301a73f0 con 0x7feb30074cd0 2024-04-23T18:24:05.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.663+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28003b50 con 0x7feb30074cd0 2024-04-23T18:24:05.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.663+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb301a78f0 con 0x7feb30074cd0 2024-04-23T18:24:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.664+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb30068d70 con 0x7feb30074cd0 2024-04-23T18:24:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.665+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb2801a070 con 0x7feb30074cd0 2024-04-23T18:24:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.666+0000 7feb2cff9640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb1c0a3d20 0x7feb1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.666+0000 7feb2f7fe640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb1c0a3d20 0x7feb1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.666+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7feb280cff70 con 0x7feb30074cd0 2024-04-23T18:24:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.666+0000 7feb2f7fe640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb1c0a3d20 0x7feb1c0a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb20009430 tx=0x7feb20009360 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:05.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.671+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb28021070 con 0x7feb30074cd0 2024-04-23T18:24:05.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 240120 0.00 MB/sec execute 10 sec latency 20678.143 ms 2024-04-23T18:24:05.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.897+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb3010a4b0 con 0x7feb30074cd0 2024-04-23T18:24:05.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.898+0000 7feb2cff9640 1 -- 172.21.15.37:0/1658647236 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11879 (secure 0 0 0) 0x7feb2809c600 con 0x7feb30074cd0 2024-04-23T18:24:05.892 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:24:05.892 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:24:03.541866+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:24:05.892 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb1c0a3d20 msgr2=0x7feb1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb1c0a3d20 0x7feb1c0a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb20009430 tx=0x7feb20009360 comp rx=0 tx=0).stop 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 msgr2=0x7feb301a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 0x7feb301a6a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb2800d950 tx=0x7feb2800de10 comp rx=0 tx=0).stop 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 shutdown_connections 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb1c0a3d20 0x7feb1c0a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb30074cd0 0x7feb301a6a60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30107280 0x7feb30074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 --2- 172.21.15.37:0/1658647236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb301021b0 0x7feb30074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:05.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 >> 172.21.15.37:0/1658647236 conn(0x7feb300fd9f0 msgr2=0x7feb300fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:05.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 shutdown_connections 2024-04-23T18:24:05.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:05.900+0000 7feb360dc640 1 -- 172.21.15.37:0/1658647236 wait complete. 2024-04-23T18:24:05.905 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 2990d434 --release --await' 2024-04-23T18:24:05.905 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id 2990d434 --release --await 2024-04-23T18:24:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 -- 172.21.15.37:0/1779209788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994102190 msgr2=0x7f2994102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 --2- 172.21.15.37:0/1779209788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994102190 0x7f2994102570 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2984009900 tx=0x7f298402a410 comp rx=0 tx=0).stop 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 -- 172.21.15.37:0/1779209788 shutdown_connections 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 --2- 172.21.15.37:0/1779209788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2994107260 0x7f299410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 --2- 172.21.15.37:0/1779209788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2994102b40 0x7f2994106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 --2- 172.21.15.37:0/1779209788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994102190 0x7f2994102570 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 -- 172.21.15.37:0/1779209788 >> 172.21.15.37:0/1779209788 conn(0x7f29940fd990 msgr2=0x7f29940ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 -- 172.21.15.37:0/1779209788 shutdown_connections 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.060+0000 7f2999f9b640 1 -- 172.21.15.37:0/1779209788 wait complete. 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 Processor -- start 2024-04-23T18:24:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 -- start start 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2994102190 0x7f2994198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 0x7f2994198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f299419cfc0 0x7f299419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29941121a0 con 0x7f2994107260 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2994112020 con 0x7f299419cfc0 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2999f9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2994112320 con 0x7f2994102190 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 0x7f2994198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 0x7f2994198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59126/0 (socket says 172.21.15.37:59126) 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 -- 172.21.15.37:0/3967434805 learned_addr learned my addr 172.21.15.37:0/3967434805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f29937fe640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2994102190 0x7f2994198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2993fff640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f299419cfc0 0x7f299419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 -- 172.21.15.37:0/3967434805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2994102190 msgr2=0x7f2994198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2994102190 0x7f2994198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 -- 172.21.15.37:0/3967434805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f299419cfc0 msgr2=0x7f299419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f299419cfc0 0x7f299419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.061+0000 7f2992ffd640 1 -- 172.21.15.37:0/3967434805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299419dab0 con 0x7f2994107260 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.062+0000 7f2992ffd640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 0x7f2994198c40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f298800d770 tx=0x7f298800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.062+0000 7f2990ff9640 1 -- 172.21.15.37:0/3967434805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988017070 con 0x7f2994107260 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.062+0000 7f2990ff9640 1 -- 172.21.15.37:0/3967434805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988009c30 con 0x7f2994107260 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.062+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299419dda0 con 0x7f2994107260 2024-04-23T18:24:06.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.062+0000 7f2990ff9640 1 -- 172.21.15.37:0/3967434805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800f620 con 0x7f2994107260 2024-04-23T18:24:06.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.062+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941a5590 con 0x7f2994107260 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.064+0000 7f2990ff9640 1 -- 172.21.15.37:0/3967434805 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f298801a070 con 0x7f2994107260 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.064+0000 7f299cd8c740 1 Processor -- start 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.064+0000 7f299cd8c740 1 -- start start 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2990ff9640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f297c0a3d20 0x7f297c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 0x56092924b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924bdb0 0x560929255f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560929256480 0x560929258870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f299cd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5609291b06d0 con 0x56092924b490 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f299cd8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56092914d1a0 con 0x56092924bdb0 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f299cd8c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5609291ac690 con 0x560929256480 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 0x56092924b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 0x56092924b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59138/0 (socket says 172.21.15.37:59138) 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 -- 172.21.15.37:0/3441456981 learned_addr learned my addr 172.21.15.37:0/3441456981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2993fff640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f297c0a3d20 0x7f297c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2990ff9640 1 -- 172.21.15.37:0/3967434805 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f29880d0380 con 0x7f2994107260 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2992ffd640 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924bdb0 0x560929255f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2993fff640 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560929256480 0x560929258870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 -- 172.21.15.37:0/3441456981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560929256480 msgr2=0x560929258870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560929256480 0x560929258870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 -- 172.21.15.37:0/3441456981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924bdb0 msgr2=0x560929255f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924bdb0 0x560929255f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 -- 172.21.15.37:0/3441456981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609291575b0 con 0x56092924b490 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f29937fe640 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 0x56092924b870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2984009900 tx=0x7f2984032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a786c1f66240e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2993fff640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f297c0a3d20 0x7f297c0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x560929248940 tx=0x7f298c00a290 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.065+0000 7f2979ffb640 1 -- 172.21.15.37:0/3441456981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984040070 con 0x56092924b490 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f2979ffb640 1 -- 172.21.15.37:0/3441456981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29840316a0 con 0x56092924b490 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f2979ffb640 1 -- 172.21.15.37:0/3441456981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984035040 con 0x56092924b490 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 -- 172.21.15.37:0/3441456981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 msgr2=0x56092924b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 0x56092924b870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2984009900 tx=0x7f2984032e60 comp rx=0 tx=0).stop 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 -- 172.21.15.37:0/3441456981 shutdown_connections 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560929256480 0x560929258870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924bdb0 0x560929255f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3441456981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924b490 0x56092924b870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 -- 172.21.15.37:0/3441456981 >> 172.21.15.37:0/3441456981 conn(0x5609292477b0 msgr2=0x560929247b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 -- 172.21.15.37:0/3441456981 shutdown_connections 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.066+0000 7f299cd8c740 1 -- 172.21.15.37:0/3441456981 wait complete. 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 -- start start 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 Processor -- start 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x560929255e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929253760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5609292592b0 con 0x560929256480 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56092914ab20 con 0x56092924b490 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560929259430 con 0x56092924bdb0 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f299cd8c740 10 client.0 _ll_register_callbacks cb 0x56092914a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f2993fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929253760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f2993fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929253760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59144/0 (socket says 172.21.15.37:59144) 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f2993fff640 1 -- 172.21.15.37:0/3992528997 learned_addr learned my addr 172.21.15.37:0/3992528997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f29937fe640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x560929255e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.067+0000 7f2992ffd640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2993fff640 1 -- 172.21.15.37:0/3992528997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 msgr2=0x560929253150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2993fff640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2993fff640 1 -- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 msgr2=0x560929255e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2993fff640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x560929255e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2993fff640 1 -- 172.21.15.37:0/3992528997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560928ec2920 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2992ffd640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2993fff640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929253760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f298c010af0 tx=0x7f298c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4097f0e66c329d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2972ffd640 1 -- 172.21.15.37:0/3992528997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c00f780 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2972ffd640 1 -- 172.21.15.37:0/3992528997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f298c00f920 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29840316a0 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2972ffd640 1 -- 172.21.15.37:0/3992528997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f298c00fb20 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f299cd8c740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560928f8ed30 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2972ffd640 1 -- 172.21.15.37:0/3992528997 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29840316a0 con 0x560929256480 2024-04-23T18:24:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f2972ffd640 1 -- 172.21.15.37:0/3992528997 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f298c01c410 con 0x560929256480 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.068+0000 7f299cd8c740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f299cd8c740 10 client.15954 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f299cd8c740 20 client.15954 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560929266690 0x560929268ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f299cd8c740 4 client.15954 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56092916e500 con 0x560929266690 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f29937fe640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560929266690 0x560929268ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f2972ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.069+0000 7f29937fe640 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560929266690 0x560929268ae0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202fd4264763694c server_cookie=1e1ed15b7a0ab77f in_seq=0 out_seq=0 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f2972ffd640 1 -- 172.21.15.37:0/3992528997 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56092916e500 con 0x560929266690 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f2972ffd640 10 client.15954 handle_command_reply: tid=0 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 client.15954 shutdown 2024-04-23T18:24:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 msgr2=0x560929253760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929253760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f298c010af0 tx=0x7f298c004ab0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 shutdown_connections 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560929266690 0x560929268ae0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929253760 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f2972ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 --2- 172.21.15.37:0/3992528997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x560929255e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 >> 172.21.15.37:0/3992528997 conn(0x5609292477b0 msgr2=0x56092924d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 shutdown_connections 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 1 -- 172.21.15.37:0/3992528997 wait complete. 2024-04-23T18:24:06.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.070+0000 7f299cd8c740 20 client.15954 trim_cache size 0 max 16384 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.087+0000 7f299cd8c740 1 Processor -- start 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.087+0000 7f299cd8c740 1 -- start start 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x56092924dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929246850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f299cd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b04a0 con 0x560929256480 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f299cd8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f29941add10 con 0x56092924b490 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f299cd8c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29941abb60 con 0x56092924bdb0 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929246850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929246850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59154/0 (socket says 172.21.15.37:59154) 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 -- 172.21.15.37:0/1802896930 learned_addr learned my addr 172.21.15.37:0/1802896930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2992ffd640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f29937fe640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x56092924dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 -- 172.21.15.37:0/1802896930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 msgr2=0x560929253340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 -- 172.21.15.37:0/1802896930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 msgr2=0x56092924dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x56092924dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 -- 172.21.15.37:0/1802896930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560929241de0 con 0x560929256480 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2992ffd640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f29937fe640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x56092924dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f2993fff640 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929246850 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f298c01c880 tx=0x7f298c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6e7778a911e43b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f29797fa640 1 -- 172.21.15.37:0/1802896930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c01c410 con 0x560929256480 2024-04-23T18:24:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f29797fa640 1 -- 172.21.15.37:0/1802896930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f298c01c5b0 con 0x560929256480 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.088+0000 7f29797fa640 1 -- 172.21.15.37:0/1802896930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c00f780 con 0x560929256480 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 -- 172.21.15.37:0/1802896930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 msgr2=0x560929246850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929246850 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f298c01c880 tx=0x7f298c004a40 comp rx=0 tx=0).stop 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 -- 172.21.15.37:0/1802896930 shutdown_connections 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560929256480 0x560929246850 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924bdb0 0x560929253340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 --2- 172.21.15.37:0/1802896930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092924b490 0x56092924dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 -- 172.21.15.37:0/1802896930 >> 172.21.15.37:0/1802896930 conn(0x5609292477b0 msgr2=0x560929246140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 -- 172.21.15.37:0/1802896930 shutdown_connections 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 -- 172.21.15.37:0/1802896930 wait complete. 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.089+0000 7f299cd8c740 1 -- start start 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 Processor -- start 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924b490 0x56092925e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 0x56092925ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560929256480 0x560929257700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29941ac1d0 con 0x56092924bdb0 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f29941ad640 con 0x560929256480 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29941ac8a0 con 0x56092924b490 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f299cd8c740 10 client.0 _ll_register_callbacks cb 0x7f298c010770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 0x56092925ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 0x56092925ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59158/0 (socket says 172.21.15.37:59158) 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 -- 172.21.15.37:0/356215195 learned_addr learned my addr 172.21.15.37:0/356215195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f29937fe640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924b490 0x56092925e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 -- 172.21.15.37:0/356215195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924b490 msgr2=0x56092925e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924b490 0x56092925e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 -- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560929256480 msgr2=0x560929257700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560929256480 0x560929257700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 -- 172.21.15.37:0/356215195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560929257c70 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f29937fe640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924b490 0x56092925e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2992ffd640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 0x56092925ebc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f298809c010 tx=0x7f2988009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b6c6c1d08bc376 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.090+0000 7f2972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f2972ffd640 1 -- 172.21.15.37:0/356215195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988021040 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f2972ffd640 1 -- 172.21.15.37:0/356215195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f29880ac170 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c01c5b0 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f2972ffd640 1 -- 172.21.15.37:0/356215195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29880ac370 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609292cd4e0 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f2972ffd640 1 -- 172.21.15.37:0/356215195 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f298c01c5b0 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f2972ffd640 1 -- 172.21.15.37:0/356215195 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2988098060 con 0x56092924bdb0 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:24:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 10 client.15960 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 20 client.15960 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5609292cdc30 0x5609292d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 4 client.15960 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2990d434", "release": true, "await": true}] 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2990d434", "release": true, "await": true}) -- 0x5609291be0f0 con 0x5609292cdc30 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.091+0000 7f29937fe640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5609292cdc30 0x5609292d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.092+0000 7f2972ffd640 10 client.15960 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.092+0000 7f29937fe640 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5609292cdc30 0x5609292d0050 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4817fe27214a6d server_cookie=bc4f9cb72c3801e5 in_seq=0 out_seq=0 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f2972ffd640 1 -- 172.21.15.37:0/356215195 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5609291be0f0 con 0x5609292cdc30 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f2972ffd640 10 client.15960 handle_command_reply: tid=0 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 client.15960 shutdown 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 msgr2=0x56092925ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 0x56092925ebc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f298809c010 tx=0x7f2988009de0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 shutdown_connections 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5609292cdc30 0x5609292d0050 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560929256480 0x560929257700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56092924bdb0 0x56092925ebc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f2972ffd640 0 client.15960 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 --2- 172.21.15.37:0/356215195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56092924b490 0x56092925e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 >> 172.21.15.37:0/356215195 conn(0x5609292477b0 msgr2=0x5609292469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 shutdown_connections 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.099+0000 7f299cd8c740 1 -- 172.21.15.37:0/356215195 wait complete. 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f299cd8c740 20 client.15960 trim_cache size 0 max 16384 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "2990d434": { 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 12, 2024-04-23T18:24:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 80.3, 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.097 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:24:06.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f297c0a3d20 msgr2=0x7f297c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f297c0a3d20 0x7f297c0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x560929248940 tx=0x7f298c00a290 comp rx=0 tx=0).stop 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 msgr2=0x7f2994198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 0x7f2994198c40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f298800d770 tx=0x7f298800dc30 comp rx=0 tx=0).stop 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 shutdown_connections 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f297c0a3d20 0x7f297c0a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f299419cfc0 0x7f299419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2994107260 0x7f2994198c40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 --2- 172.21.15.37:0/3967434805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2994102190 0x7f2994198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 >> 172.21.15.37:0/3967434805 conn(0x7f29940fd990 msgr2=0x7f29940fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 shutdown_connections 2024-04-23T18:24:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.100+0000 7f2999f9b640 1 -- 172.21.15.37:0/3967434805 wait complete. 2024-04-23T18:24:06.105 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2990d434', seconds elapsed: 21.6 2024-04-23T18:24:06.105 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-04-23T18:24:06.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 -- 172.21.15.37:0/2515839531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a410c520 msgr2=0x7fe6a410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 --2- 172.21.15.37:0/2515839531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a410c520 0x7fe6a410e9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009ca0 tx=0x7fe69c02a9e0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 -- 172.21.15.37:0/2515839531 shutdown_connections 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 --2- 172.21.15.37:0/2515839531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a410c520 0x7fe6a410e9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 --2- 172.21.15.37:0/2515839531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6a4103b00 0x7fe6a410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 --2- 172.21.15.37:0/2515839531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6a41031e0 0x7fe6a41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 -- 172.21.15.37:0/2515839531 >> 172.21.15.37:0/2515839531 conn(0x7fe6a4077f50 msgr2=0x7fe6a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 -- 172.21.15.37:0/2515839531 shutdown_connections 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.230+0000 7fe6a9089640 1 -- 172.21.15.37:0/2515839531 wait complete. 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 Processor -- start 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 -- start start 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6a41031e0 0x7fe6a4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 0x7fe6a4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6a410c520 0x7fe6a419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41121e0 con 0x7fe6a4103b00 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4112060 con 0x7fe6a410c520 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a9089640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4112360 con 0x7fe6a41031e0 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 0x7fe6a4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 0x7fe6a4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59182/0 (socket says 172.21.15.37:59182) 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 -- 172.21.15.37:0/740865579 learned_addr learned my addr 172.21.15.37:0/740865579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6a410c520 0x7fe6a419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a2d9d640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6a41031e0 0x7fe6a4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 -- 172.21.15.37:0/740865579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6a41031e0 msgr2=0x7fe6a4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6a41031e0 0x7fe6a4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 -- 172.21.15.37:0/740865579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6a410c520 msgr2=0x7fe6a419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6a410c520 0x7fe6a419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.231+0000 7fe6a259c640 1 -- 172.21.15.37:0/740865579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a419d600 con 0x7fe6a4103b00 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.232+0000 7fe6a259c640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 0x7fe6a4198ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c920 tx=0x7fe69400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.232+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694007c70 con 0x7fe6a4103b00 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.232+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a419d8f0 con 0x7fe6a4103b00 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.232+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe694007e10 con 0x7fe6a4103b00 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.232+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694018820 con 0x7fe6a4103b00 2024-04-23T18:24:06.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.232+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a5180 con 0x7fe6a4103b00 2024-04-23T18:24:06.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.233+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a4068d70 con 0x7fe6a4103b00 2024-04-23T18:24:06.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.234+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe69400f450 con 0x7fe6a4103b00 2024-04-23T18:24:06.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.235+0000 7fe69bfff640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe6900a3d20 0x7fe6900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.235+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe6940d0ab0 con 0x7fe6a4103b00 2024-04-23T18:24:06.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.235+0000 7fe6a2d9d640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe6900a3d20 0x7fe6900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.235+0000 7fe6a2d9d640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe6900a3d20 0x7fe6900a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009bd0 tx=0x7fe68c005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.239+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe694014030 con 0x7fe6a4103b00 2024-04-23T18:24:06.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.478+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6a4199960 con 0x7fe6a4103b00 2024-04-23T18:24:06.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.479+0000 7fe69bfff640 1 -- 172.21.15.37:0/740865579 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11879 (secure 0 0 0) 0x7fe69409d1d0 con 0x7fe6a4103b00 2024-04-23T18:24:06.473 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:24:06.473 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:24:03.541866+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:24:06.474 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-04-23T18:24:06.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe6900a3d20 msgr2=0x7fe6900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe6900a3d20 0x7fe6900a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009bd0 tx=0x7fe68c005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 msgr2=0x7fe6a4198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 0x7fe6a4198ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c920 tx=0x7fe69400cde0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 shutdown_connections 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe6900a3d20 0x7fe6900a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6a410c520 0x7fe6a419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4103b00 0x7fe6a4198ba0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 --2- 172.21.15.37:0/740865579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6a41031e0 0x7fe6a4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.481+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 >> 172.21.15.37:0/740865579 conn(0x7fe6a4077f50 msgr2=0x7fe6a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.482+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 shutdown_connections 2024-04-23T18:24:06.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.482+0000 7fe6a9089640 1 -- 172.21.15.37:0/740865579 wait complete. 2024-04-23T18:24:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:06 smithi037 ceph-mon[29404]: pgmap v112: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-04-23T18:24:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:06 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1658647236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:06.487 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 80.35 --await-for 42.0' 2024-04-23T18:24:06.487 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 80.35 --await-for 42.0 2024-04-23T18:24:06.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:06 smithi172 ceph-mon[32456]: pgmap v112: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-04-23T18:24:06.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:06 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1658647236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 -- 172.21.15.37:0/2524055196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c107260 msgr2=0x7fb62c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 --2- 172.21.15.37:0/2524055196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c107260 0x7fb62c10ee30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb624009ca0 tx=0x7fb62402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 -- 172.21.15.37:0/2524055196 shutdown_connections 2024-04-23T18:24:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 --2- 172.21.15.37:0/2524055196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c107260 0x7fb62c10ee30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 --2- 172.21.15.37:0/2524055196 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb62c102b40 0x7fb62c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 --2- 172.21.15.37:0/2524055196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c102190 0x7fb62c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 -- 172.21.15.37:0/2524055196 >> 172.21.15.37:0/2524055196 conn(0x7fb62c079310 msgr2=0x7fb62c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.608+0000 7fb633b4f640 1 -- 172.21.15.37:0/2524055196 shutdown_connections 2024-04-23T18:24:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 -- 172.21.15.37:0/2524055196 wait complete. 2024-04-23T18:24:06.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 Processor -- start 2024-04-23T18:24:06.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 -- start start 2024-04-23T18:24:06.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 0x7fb62c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c107260 0x7fb62c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb62c19cfc0 0x7fb62c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1121b0 con 0x7fb62c102190 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c112030 con 0x7fb62c107260 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb633b4f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c112330 con 0x7fb62c19cfc0 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.609+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 0x7fb62c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 0x7fb62c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59192/0 (socket says 172.21.15.37:59192) 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6318eb640 1 -- 172.21.15.37:0/1119008513 learned_addr learned my addr 172.21.15.37:0/1119008513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c107260 0x7fb62c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6320ec640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb62c19cfc0 0x7fb62c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 -- 172.21.15.37:0/1119008513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb62c19cfc0 msgr2=0x7fb62c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb62c19cfc0 0x7fb62c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 -- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 msgr2=0x7fb62c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 0x7fb62c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 -- 172.21.15.37:0/1119008513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c19dab0 con 0x7fb62c107260 2024-04-23T18:24:06.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6318eb640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 0x7fb62c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:06.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb6310ea640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c107260 0x7fb62c198c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00e9c0 tx=0x7fb61c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb622ffd640 1 -- 172.21.15.37:0/1119008513 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00ccc0 con 0x7fb62c107260 2024-04-23T18:24:06.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c19dda0 con 0x7fb62c107260 2024-04-23T18:24:06.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb622ffd640 1 -- 172.21.15.37:0/1119008513 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c0108e0 con 0x7fb62c107260 2024-04-23T18:24:06.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb622ffd640 1 -- 172.21.15.37:0/1119008513 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0187a0 con 0x7fb62c107260 2024-04-23T18:24:06.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.610+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a5590 con 0x7fb62c107260 2024-04-23T18:24:06.608 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:06 smithi173 ceph-mon[32906]: pgmap v112: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-04-23T18:24:06.608 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:06 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1658647236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.612+0000 7fb622ffd640 1 -- 172.21.15.37:0/1119008513 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb61c010430 con 0x7fb62c107260 2024-04-23T18:24:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 Processor -- start 2024-04-23T18:24:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 -- start start 2024-04-23T18:24:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed1884a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18854b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed18857480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ed187af370 con 0x55ed1884a0a0 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ed1874c2e0 con 0x55ed18855090 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.613+0000 7fb63686f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ed187ab330 con 0x55ed1884a9c0 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18854b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6320ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed1884a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb622ffd640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb6180a3d20 0x7fb6180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6310ea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed18857480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6320ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed1884a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59202/0 (socket says 172.21.15.37:59202) 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18854b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:54278/0 (socket says 172.21.15.37:54278) 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6320ec640 1 -- 172.21.15.37:0/3259372048 learned_addr learned my addr 172.21.15.37:0/3259372048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6320ec640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb6180a3d20 0x7fb6180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 -- 172.21.15.37:0/3259372048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 msgr2=0x55ed18857480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed18857480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 -- 172.21.15.37:0/3259372048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 msgr2=0x55ed1884a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed1884a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb622ffd640 1 -- 172.21.15.37:0/1119008513 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fb61c0d0af0 con 0x7fb62c107260 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 -- 172.21.15.37:0/3259372048 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1858dd80 con 0x55ed1884a9c0 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6320ec640 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed1884a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6318eb640 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18854b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6280114f0 tx=0x7fb6280119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc3148bd3e7b1f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb6320ec640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb6180a3d20 0x7fb6180a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb62402f500 tx=0x7fb62402a8c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb617fff640 1 -- 172.21.15.37:0/3259372048 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62801e070 con 0x55ed1884a9c0 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb617fff640 1 -- 172.21.15.37:0/3259372048 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6280035f0 con 0x55ed1884a9c0 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.614+0000 7fb617fff640 1 -- 172.21.15.37:0/3259372048 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62800f820 con 0x55ed1884a9c0 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 -- 172.21.15.37:0/3259372048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 msgr2=0x55ed18854b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18854b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6280114f0 tx=0x7fb6280119b0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 -- 172.21.15.37:0/3259372048 shutdown_connections 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed18857480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18854b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 --2- 172.21.15.37:0/3259372048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed1884a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 -- 172.21.15.37:0/3259372048 >> 172.21.15.37:0/3259372048 conn(0x55ed188463c0 msgr2=0x55ed188467a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 -- 172.21.15.37:0/3259372048 shutdown_connections 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.615+0000 7fb63686f740 1 -- 172.21.15.37:0/3259372048 wait complete. 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 -- start start 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 Processor -- start 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 0x55ed18851fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed188524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18852a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ed18857e80 con 0x55ed18855090 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ed18749fb0 con 0x55ed1884a0a0 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ed18858000 con 0x55ed1884a9c0 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb63686f740 10 client.0 _ll_register_callbacks cb 0x55ed187499a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6320ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18852a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 0x55ed18851fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6310ea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed188524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 0x55ed18851fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:53506/0 (socket says 172.21.15.37:53506) 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6310ea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed188524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:54288/0 (socket says 172.21.15.37:54288) 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6318eb640 1 -- 172.21.15.37:0/2108900996 learned_addr learned my addr 172.21.15.37:0/2108900996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.616+0000 7fb6320ec640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18852a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59216/0 (socket says 172.21.15.37:59216) 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6318eb640 1 -- 172.21.15.37:0/2108900996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 msgr2=0x55ed188524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6318eb640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed188524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6318eb640 1 -- 172.21.15.37:0/2108900996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 msgr2=0x55ed18852a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6318eb640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18852a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6318eb640 1 -- 172.21.15.37:0/2108900996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed185ee510 con 0x55ed1884a0a0 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6310ea640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed188524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6320ec640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18852a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:06.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6318eb640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 0x55ed18851fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6280114c0 tx=0x7fb628004200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b7c2ca63cd4e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6157fa640 1 -- 172.21.15.37:0/2108900996 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62801e030 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6157fa640 1 -- 172.21.15.37:0/2108900996 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fb628005050 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6280035f0 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6157fa640 1 -- 172.21.15.37:0/2108900996 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb628031050 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb63686f740 10 client.25538 fetch_fsmap learned FSMap version 29 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed18481670 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.617+0000 7fb6157fa640 1 -- 172.21.15.37:0/2108900996 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6280035f0 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb6157fa640 1 -- 172.21.15.37:0/2108900996 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fb62800f5c0 con 0x55ed1884a0a0 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb63686f740 10 client.25538 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb63686f740 10 client.25538 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb63686f740 20 client.25538 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb63686f740 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb63686f740 4 client.25538 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed187bcd90 con 0x55ed188652f0 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb6310ea640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb6310ea640 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188676e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6288b137ee344ddb server_cookie=851c4294f03fd227 in_seq=0 out_seq=0 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.618+0000 7fb6157fa640 10 client.25538 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb6157fa640 1 -- 172.21.15.37:0/2108900996 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed187bcd90 con 0x55ed188652f0 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb6157fa640 10 client.25538 handle_command_reply: tid=0 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 client.25538 shutdown 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 msgr2=0x55ed18851fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 0x55ed18851fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6280114c0 tx=0x7fb628004200 comp rx=0 tx=0).stop 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 shutdown_connections 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188676e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18852a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb6157fa640 0 client.25538 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed188524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 --2- 172.21.15.37:0/2108900996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a0a0 0x55ed18851fb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 >> 172.21.15.37:0/2108900996 conn(0x55ed188463c0 msgr2=0x55ed1884c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 shutdown_connections 2024-04-23T18:24:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 1 -- 172.21.15.37:0/2108900996 wait complete. 2024-04-23T18:24:06.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.619+0000 7fb63686f740 20 client.25538 trim_cache size 0 max 16384 2024-04-23T18:24:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 Processor -- start 2024-04-23T18:24:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 -- start start 2024-04-23T18:24:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a0a0 0x55ed1884c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a9c0 0x55ed18860d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18845480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1b05f0 con 0x55ed18855090 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ade60 con 0x55ed1884a9c0 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.638+0000 7fb63686f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1abcb0 con 0x55ed1884a0a0 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18845480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18845480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59224/0 (socket says 172.21.15.37:59224) 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 -- 172.21.15.37:0/3826772630 learned_addr learned my addr 172.21.15.37:0/3826772630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6318eb640 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a0a0 0x55ed1884c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6310ea640 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a9c0 0x55ed18860d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 -- 172.21.15.37:0/3826772630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a0a0 msgr2=0x55ed1884c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a0a0 0x55ed1884c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 -- 172.21.15.37:0/3826772630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a9c0 msgr2=0x55ed18860d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a9c0 0x55ed18860d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 -- 172.21.15.37:0/3826772630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed18867a50 con 0x55ed18855090 2024-04-23T18:24:06.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6320ec640 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18845480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb624032ff0 tx=0x7fb624045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4e34109143b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6177fe640 1 -- 172.21.15.37:0/3826772630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624055070 con 0x55ed18855090 2024-04-23T18:24:06.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6177fe640 1 -- 172.21.15.37:0/3826772630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb624036050 con 0x55ed18855090 2024-04-23T18:24:06.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.639+0000 7fb6177fe640 1 -- 172.21.15.37:0/3826772630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62402fa80 con 0x55ed18855090 2024-04-23T18:24:06.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 -- 172.21.15.37:0/3826772630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 msgr2=0x55ed18845480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18845480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb624032ff0 tx=0x7fb624045000 comp rx=0 tx=0).stop 2024-04-23T18:24:06.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 -- 172.21.15.37:0/3826772630 shutdown_connections 2024-04-23T18:24:06.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed18855090 0x55ed18845480 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1884a9c0 0x55ed18860d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 --2- 172.21.15.37:0/3826772630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a0a0 0x55ed1884c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 -- 172.21.15.37:0/3826772630 >> 172.21.15.37:0/3826772630 conn(0x55ed188463c0 msgr2=0x55ed18844d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 -- 172.21.15.37:0/3826772630 shutdown_connections 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 -- 172.21.15.37:0/3826772630 wait complete. 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 -- start start 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.640+0000 7fb63686f740 1 Processor -- start 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed18858400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18858940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed188c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ac320 con 0x55ed1884a0a0 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ad790 con 0x55ed18855090 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ac9f0 con 0x55ed1884a9c0 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 10 client.0 _ll_register_callbacks cb 0x7fb61c09c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:06.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed18858400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed18858400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59226/0 (socket says 172.21.15.37:59226) 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 -- 172.21.15.37:0/2224499694 learned_addr learned my addr 172.21.15.37:0/2224499694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6310ea640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18858940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6320ec640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed188c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 -- 172.21.15.37:0/2224499694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 msgr2=0x55ed18858940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18858940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 -- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 msgr2=0x55ed188c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed188c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 -- 172.21.15.37:0/2224499694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed188c9d10 con 0x55ed1884a0a0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6318eb640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed18858400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb62800fbd0 tx=0x7fb62800f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88f252039637976 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6157fa640 1 -- 172.21.15.37:0/2224499694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62803a060 con 0x55ed1884a0a0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb6157fa640 1 -- 172.21.15.37:0/2224499694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fb628004600 con 0x55ed1884a0a0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.641+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb624036050 con 0x55ed1884a0a0 2024-04-23T18:24:06.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb6157fa640 1 -- 172.21.15.37:0/2224499694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb62801e070 con 0x55ed1884a0a0 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 10 client.15984 fetch_fsmap learned FSMap version 29 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed188cafd0 con 0x55ed1884a0a0 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb6157fa640 1 -- 172.21.15.37:0/2224499694 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb624036050 con 0x55ed1884a0a0 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb6157fa640 1 -- 172.21.15.37:0/2224499694 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fb628019900 con 0x55ed1884a0a0 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 10 client.15984 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 20 client.15984 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 4 client.15984 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.35, "await_for": 42.0}] 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.35, "await_for": 42.0}) -- 0x55ed187b6c00 con 0x55ed188652f0 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.642+0000 7fb6310ea640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.643+0000 7fb6157fa640 10 client.15984 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.643+0000 7fb6310ea640 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188cd7a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4d131ed512cece server_cookie=14100bf902c43735 in_seq=0 out_seq=0 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb6157fa640 1 -- 172.21.15.37:0/2224499694 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ed187b6c00 con 0x55ed188652f0 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb6157fa640 10 client.15984 handle_command_reply: tid=0 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 client.15984 shutdown 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 msgr2=0x55ed18858400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed18858400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb62800fbd0 tx=0x7fb62800f430 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 shutdown_connections 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55ed188652f0 0x55ed188cd7a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed18855090 0x55ed188c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed1884a9c0 0x55ed18858940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb6157fa640 0 client.15984 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 --2- 172.21.15.37:0/2224499694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ed1884a0a0 0x55ed18858400 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 >> 172.21.15.37:0/2224499694 conn(0x55ed188463c0 msgr2=0x55ed18845600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.727+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 shutdown_connections 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb63686f740 1 -- 172.21.15.37:0/2224499694 wait complete. 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb63686f740 20 client.15984 trim_cache size 0 max 16384 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 14, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "c72b2d62": { 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 14, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 80.3, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:24:06.724 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb6180a3d20 msgr2=0x7fb6180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb6180a3d20 0x7fb6180a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb62402f500 tx=0x7fb62402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c107260 msgr2=0x7fb62c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c107260 0x7fb62c198c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00e9c0 tx=0x7fb61c00ee80 comp rx=0 tx=0).stop 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 shutdown_connections 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb6180a3d20 0x7fb6180a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb62c19cfc0 0x7fb62c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb62c107260 0x7fb62c198c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 --2- 172.21.15.37:0/1119008513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c102190 0x7fb62c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 >> 172.21.15.37:0/1119008513 conn(0x7fb62c079310 msgr2=0x7fb62c100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 shutdown_connections 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:06.728+0000 7fb633b4f640 1 -- 172.21.15.37:0/1119008513 wait complete. 2024-04-23T18:24:06.733 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c72b2d62', quiesce took 0.6 seconds. Will release after: 20.35 2024-04-23T18:24:06.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 1.06 MB/sec execute 11 sec latency 20961.320 ms 2024-04-23T18:24:07.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:07 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/740865579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:07.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:07 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/740865579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:07.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:07 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/740865579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:07.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.97 MB/sec execute 12 sec latency 1151.302 ms 2024-04-23T18:24:08.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:08 smithi172 ceph-mon[32456]: pgmap v113: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 2.4 MiB/s wr, 120 op/s 2024-04-23T18:24:08.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:08 smithi173 ceph-mon[32906]: pgmap v113: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 2.4 MiB/s wr, 120 op/s 2024-04-23T18:24:08.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:08 smithi037 ceph-mon[29404]: pgmap v113: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 2.4 MiB/s wr, 120 op/s 2024-04-23T18:24:08.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.90 MB/sec execute 13 sec latency 2151.422 ms 2024-04-23T18:24:09.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.83 MB/sec execute 14 sec latency 3151.528 ms 2024-04-23T18:24:10.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:10 smithi172 ceph-mon[32456]: pgmap v114: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-04-23T18:24:10.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:10 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:10.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:10 smithi173 ceph-mon[32906]: pgmap v114: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-04-23T18:24:10.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:10 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:10.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:10 smithi037 ceph-mon[29404]: pgmap v114: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-04-23T18:24:10.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:10 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:10.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.78 MB/sec execute 15 sec latency 4151.617 ms 2024-04-23T18:24:11.814 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.73 MB/sec execute 16 sec latency 5151.749 ms 2024-04-23T18:24:12.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:12 smithi172 ceph-mon[32456]: pgmap v115: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-04-23T18:24:12.605 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:12 smithi173 ceph-mon[32906]: pgmap v115: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-04-23T18:24:12.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:12 smithi037 ceph-mon[29404]: pgmap v115: 97 pgs: 97 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.7 MiB/s wr, 101 op/s 2024-04-23T18:24:12.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.69 MB/sec execute 17 sec latency 6151.850 ms 2024-04-23T18:24:13.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.65 MB/sec execute 18 sec latency 7151.918 ms 2024-04-23T18:24:14.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:14 smithi172 ceph-mon[32456]: pgmap v116: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.4 MiB/s wr, 128 op/s 2024-04-23T18:24:14.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:14 smithi173 ceph-mon[32906]: pgmap v116: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.4 MiB/s wr, 128 op/s 2024-04-23T18:24:14.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:14 smithi037 ceph-mon[29404]: pgmap v116: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.4 MiB/s wr, 128 op/s 2024-04-23T18:24:14.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.61 MB/sec execute 19 sec latency 8152.022 ms 2024-04-23T18:24:15.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:15 smithi172 ceph-mon[32456]: pgmap v117: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-04-23T18:24:15.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:15 smithi173 ceph-mon[32906]: pgmap v117: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-04-23T18:24:15.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:15 smithi037 ceph-mon[29404]: pgmap v117: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-04-23T18:24:15.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.58 MB/sec execute 20 sec latency 9152.112 ms 2024-04-23T18:24:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 -- 172.21.15.37:0/2494112152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 msgr2=0x7f2a6c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 --2- 172.21.15.37:0/2494112152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c110bd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009ca0 tx=0x7f2a6402a950 comp rx=0 tx=0).stop 2024-04-23T18:24:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 -- 172.21.15.37:0/2494112152 shutdown_connections 2024-04-23T18:24:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 --2- 172.21.15.37:0/2494112152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c110bd0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 --2- 172.21.15.37:0/2494112152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c074ca0 0x7f2a6c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 --2- 172.21.15.37:0/2494112152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a6c074380 0x7f2a6c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 -- 172.21.15.37:0/2494112152 >> 172.21.15.37:0/2494112152 conn(0x7f2a6c0fda10 msgr2=0x7f2a6c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 -- 172.21.15.37:0/2494112152 shutdown_connections 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.655+0000 7f2a73720640 1 -- 172.21.15.37:0/2494112152 wait complete. 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 Processor -- start 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 -- start start 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c074380 0x7f2a6c101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a6c074ca0 0x7f2a6c1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1143b0 con 0x7f2a6c10e780 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c114230 con 0x7f2a6c074ca0 2024-04-23T18:24:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a73720640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c114530 con 0x7f2a6c074380 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a71cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a71cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36008/0 (socket says 172.21.15.37:36008) 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a71cbd640 1 -- 172.21.15.37:0/704592525 learned_addr learned my addr 172.21.15.37:0/704592525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.656+0000 7f2a714bc640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c074380 0x7f2a6c101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a6c074ca0 0x7f2a6c1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a71cbd640 1 -- 172.21.15.37:0/704592525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c074380 msgr2=0x7f2a6c101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a71cbd640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c074380 0x7f2a6c101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a71cbd640 1 -- 172.21.15.37:0/704592525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a6c074ca0 msgr2=0x7f2a6c1023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a71cbd640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a6c074ca0 0x7f2a6c1023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a71cbd640 1 -- 172.21.15.37:0/704592525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1aaf00 con 0x7f2a6c10e780 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a71cbd640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c102920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2a64006010 tx=0x7f2a6402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a627fc640 1 -- 172.21.15.37:0/704592525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64034040 con 0x7f2a6c10e780 2024-04-23T18:24:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a627fc640 1 -- 172.21.15.37:0/704592525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a640308f0 con 0x7f2a6c10e780 2024-04-23T18:24:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1ab190 con 0x7f2a6c10e780 2024-04-23T18:24:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a627fc640 1 -- 172.21.15.37:0/704592525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640047f0 con 0x7f2a6c10e780 2024-04-23T18:24:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.657+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1ab680 con 0x7f2a6c10e780 2024-04-23T18:24:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a627fc640 1 -- 172.21.15.37:0/704592525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a64037020 con 0x7f2a6c10e780 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 Processor -- start 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 -- start start 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2323a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323f40 0x55a7f232e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f232e800 0x55a7f2330bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2288970 con 0x55a7f2323620 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7f22254a0 con 0x55a7f232e800 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a76435740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2284930 con 0x55a7f2323f40 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.660+0000 7f2a627fc640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2a580a3dc0 0x7f2a580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2323a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2323a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36018/0 (socket says 172.21.15.37:36018) 2024-04-23T18:24:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 -- 172.21.15.37:0/1785721101 learned_addr learned my addr 172.21.15.37:0/1785721101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a71cbd640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2a580a3dc0 0x7f2a580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a714bc640 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323f40 0x55a7f232e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a627fc640 1 -- 172.21.15.37:0/704592525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2a640f16a0 con 0x7f2a6c10e780 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f232e800 0x55a7f2330bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 -- 172.21.15.37:0/1785721101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323f40 msgr2=0x55a7f232e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323f40 0x55a7f232e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 -- 172.21.15.37:0/1785721101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f232e800 msgr2=0x55a7f2330bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f232e800 0x55a7f2330bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 -- 172.21.15.37:0/1785721101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f1f9b210 con 0x55a7f2323620 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a714bc640 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323f40 0x55a7f232e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2323a00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0058c0 tx=0x7f2a5c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94b88a236c47853 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a71cbd640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2a580a3dc0 0x7f2a580a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a640eefa0 tx=0x7f2a64087d40 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a577fe640 1 -- 172.21.15.37:0/1785721101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017070 con 0x55a7f2323620 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a577fe640 1 -- 172.21.15.37:0/1785721101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c0138b0 con 0x55a7f2323620 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.661+0000 7f2a577fe640 1 -- 172.21.15.37:0/1785721101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c012680 con 0x55a7f2323620 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 -- 172.21.15.37:0/1785721101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 msgr2=0x55a7f2323a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2323a00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0058c0 tx=0x7f2a5c0094c0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 -- 172.21.15.37:0/1785721101 shutdown_connections 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f232e800 0x55a7f2330bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323f40 0x55a7f232e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 --2- 172.21.15.37:0/1785721101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2323a00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 -- 172.21.15.37:0/1785721101 >> 172.21.15.37:0/1785721101 conn(0x55a7f231fb30 msgr2=0x55a7f231ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 -- 172.21.15.37:0/1785721101 shutdown_connections 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 -- 172.21.15.37:0/1785721101 wait complete. 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.662+0000 7f2a76435740 1 -- start start 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 Processor -- start 2024-04-23T18:24:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323620 0x55a7f232e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 0x55a7f232b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f232ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2331430 con 0x55a7f2323f40 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7f22230b0 con 0x55a7f2323620 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7f23315b0 con 0x55a7f232e800 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a76435740 10 client.0 _ll_register_callbacks cb 0x55a7f2210d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 0x55a7f232b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 0x55a7f232b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36024/0 (socket says 172.21.15.37:36024) 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 -- 172.21.15.37:0/2921359076 learned_addr learned my addr 172.21.15.37:0/2921359076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f232ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323620 0x55a7f232e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 -- 172.21.15.37:0/2921359076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 msgr2=0x55a7f232ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f232ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 -- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323620 msgr2=0x55a7f232e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323620 0x55a7f232e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.663+0000 7f2a714bc640 1 -- 172.21.15.37:0/2921359076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f2067620 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 0x55a7f232b4a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800bef0 tx=0x7f2a68012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae027ef913b4dea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2921359076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68005220 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2921359076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2a680053c0 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c0138b0 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2921359076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a68005600 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a76435740 10 client.15996 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7f233d400 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2921359076 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c0138b0 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2921359076 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2a68010c20 con 0x55a7f2323f40 2024-04-23T18:24:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a76435740 10 client.15996 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a76435740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.664+0000 7f2a76435740 20 client.15996 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.665+0000 7f2a76435740 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f233e910 0x55a7f2340d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.665+0000 7f2a76435740 4 client.15996 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.665+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f2296390 con 0x55a7f233e910 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.665+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f233e910 0x55a7f2340d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.665+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f233e910 0x55a7f2340d20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe7b2c4c976eb83 server_cookie=b8291b7bc4f52ff0 in_seq=0 out_seq=0 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.665+0000 7f2a54ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2921359076 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7f2296390 con 0x55a7f233e910 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a54ff9640 10 client.15996 handle_command_reply: tid=0 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 client.15996 shutdown 2024-04-23T18:24:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 msgr2=0x55a7f232b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 0x55a7f232b4a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800bef0 tx=0x7f2a68012820 comp rx=0 tx=0).stop 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 shutdown_connections 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f233e910 0x55a7f2340d20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f232ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323f40 0x55a7f232b4a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a54ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 --2- 172.21.15.37:0/2921359076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323620 0x55a7f232e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 >> 172.21.15.37:0/2921359076 conn(0x55a7f231fb30 msgr2=0x55a7f23258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 shutdown_connections 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 1 -- 172.21.15.37:0/2921359076 wait complete. 2024-04-23T18:24:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.666+0000 7f2a76435740 20 client.15996 trim_cache size 0 max 16384 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 Processor -- start 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 -- start start 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323620 0x55a7f232dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f2336800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 0x55a7f2336d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b69a0 con 0x55a7f232e800 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b4210 con 0x55a7f2323f40 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a76435740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b2060 con 0x55a7f2323620 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a70cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 0x55a7f2336d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a70cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 0x55a7f2336d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36032/0 (socket says 172.21.15.37:36032) 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a70cbb640 1 -- 172.21.15.37:0/3964132996 learned_addr learned my addr 172.21.15.37:0/3964132996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323620 0x55a7f232dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.682+0000 7f2a714bc640 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f2336800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a70cbb640 1 -- 172.21.15.37:0/3964132996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323620 msgr2=0x55a7f232dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323620 0x55a7f232dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a70cbb640 1 -- 172.21.15.37:0/3964132996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 msgr2=0x55a7f2336800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f2336800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a70cbb640 1 -- 172.21.15.37:0/3964132996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f232bdb0 con 0x55a7f232e800 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 0x55a7f2336d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00a010 tx=0x7f2a5c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf8a8e75e521872 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a56ffd640 1 -- 172.21.15.37:0/3964132996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017040 con 0x55a7f232e800 2024-04-23T18:24:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a56ffd640 1 -- 172.21.15.37:0/3964132996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c02e900 con 0x55a7f232e800 2024-04-23T18:24:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.683+0000 7f2a56ffd640 1 -- 172.21.15.37:0/3964132996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c01ccf0 con 0x55a7f232e800 2024-04-23T18:24:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 -- 172.21.15.37:0/3964132996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 msgr2=0x55a7f2336d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 0x55a7f2336d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00a010 tx=0x7f2a5c005850 comp rx=0 tx=0).stop 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 -- 172.21.15.37:0/3964132996 shutdown_connections 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f232e800 0x55a7f2336d40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f2336800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 --2- 172.21.15.37:0/3964132996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f2323620 0x55a7f232dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 -- 172.21.15.37:0/3964132996 >> 172.21.15.37:0/3964132996 conn(0x55a7f231fb30 msgr2=0x55a7f23392e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 -- 172.21.15.37:0/3964132996 shutdown_connections 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 -- 172.21.15.37:0/3964132996 wait complete. 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 -- start start 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.684+0000 7f2a76435740 1 Processor -- start 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2334480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f23349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f2334f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b26d0 con 0x55a7f2323620 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b3b40 con 0x55a7f2323f40 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b2da0 con 0x55a7f232e800 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 10 client.0 _ll_register_callbacks cb 0x7f2a6800b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2334480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2334480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36048/0 (socket says 172.21.15.37:36048) 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 -- 172.21.15.37:0/2957740211 learned_addr learned my addr 172.21.15.37:0/2957740211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f23349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 -- 172.21.15.37:0/2957740211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 msgr2=0x55a7f2334f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f2334f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 -- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 msgr2=0x55a7f23349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f23349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 -- 172.21.15.37:0/2957740211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f23a4100 con 0x55a7f2323620 2024-04-23T18:24:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f23349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a70cbb640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2334480 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c010b70 tx=0x7f2a5c010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596dced10a941dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2957740211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017050 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2957740211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2a5c02e650 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.685+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c02e900 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2957740211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c043050 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 10 client.16002 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7f23a53f0 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2957740211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c02e900 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2957740211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2a5c02b750 con 0x55a7f2323620 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 10 client.16002 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 20 client.16002 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f23a58b0 0x55a7f23a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 4 client.16002 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7f2290200 con 0x55a7f23a58b0 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.686+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f23a58b0 0x55a7f23a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.687+0000 7f2a54ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.687+0000 7f2a714bc640 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f23a58b0 0x55a7f23a7ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1e84f8d2de3b86 server_cookie=2cf96794a0199fdb in_seq=0 out_seq=0 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a54ff9640 1 -- 172.21.15.37:0/2957740211 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7f2290200 con 0x55a7f23a58b0 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a54ff9640 10 client.16002 handle_command_reply: tid=0 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 client.16002 shutdown 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 msgr2=0x55a7f2334480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2334480 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c010b70 tx=0x7f2a5c010d90 comp rx=0 tx=0).stop 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 shutdown_connections 2024-04-23T18:24:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7f23a58b0 0x55a7f23a7ca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7f232e800 0x55a7f2334f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7f2323f40 0x55a7f23349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a54ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 --2- 172.21.15.37:0/2957740211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7f2323620 0x55a7f2334480 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 >> 172.21.15.37:0/2957740211 conn(0x55a7f231fb30 msgr2=0x55a7f2339b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 shutdown_connections 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 1 -- 172.21.15.37:0/2957740211 wait complete. 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a76435740 20 client.16002 trim_cache size 0 max 16384 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:24:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:24:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:24:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2a580a3dc0 msgr2=0x7f2a580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a73720640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2a580a3dc0 0x7f2a580a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a640eefa0 tx=0x7f2a64087d40 comp rx=0 tx=0).stop 2024-04-23T18:24:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 msgr2=0x7f2a6c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.710+0000 7f2a73720640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c102920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2a64006010 tx=0x7f2a6402f5d0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 shutdown_connections 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f2a580a3dc0 0x7f2a580a6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c10e780 0x7f2a6c102920 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a6c074ca0 0x7f2a6c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 --2- 172.21.15.37:0/704592525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c074380 0x7f2a6c101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 >> 172.21.15.37:0/704592525 conn(0x7f2a6c0fda10 msgr2=0x7f2a6c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 shutdown_connections 2024-04-23T18:24:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:16.711+0000 7f2a73720640 1 -- 172.21.15.37:0/704592525 wait complete. 2024-04-23T18:24:16.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.56 MB/sec execute 21 sec latency 10152.229 ms 2024-04-23T18:24:17.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.53 MB/sec execute 22 sec latency 11152.319 ms 2024-04-23T18:24:18.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:17 smithi172 ceph-mon[32456]: pgmap v118: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.4 MiB/s wr, 115 op/s 2024-04-23T18:24:18.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:17 smithi173 ceph-mon[32906]: pgmap v118: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.4 MiB/s wr, 115 op/s 2024-04-23T18:24:18.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:17 smithi037 ceph-mon[29404]: pgmap v118: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.4 MiB/s wr, 115 op/s 2024-04-23T18:24:18.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.51 MB/sec execute 23 sec latency 12152.432 ms 2024-04-23T18:24:19.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.49 MB/sec execute 24 sec latency 13152.562 ms 2024-04-23T18:24:20.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:19 smithi172 ceph-mon[32456]: pgmap v119: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-04-23T18:24:20.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:19 smithi173 ceph-mon[32906]: pgmap v119: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-04-23T18:24:20.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:19 smithi037 ceph-mon[29404]: pgmap v119: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-04-23T18:24:20.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.47 MB/sec execute 25 sec latency 14152.669 ms 2024-04-23T18:24:21.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.45 MB/sec execute 26 sec latency 15152.814 ms 2024-04-23T18:24:22.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:21 smithi172 ceph-mon[32456]: pgmap v120: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-04-23T18:24:22.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:21 smithi173 ceph-mon[32906]: pgmap v120: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-04-23T18:24:22.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:21 smithi037 ceph-mon[29404]: pgmap v120: 97 pgs: 97 active+clean; 123 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-04-23T18:24:22.815 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.43 MB/sec execute 27 sec latency 16152.901 ms 2024-04-23T18:24:23.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.42 MB/sec execute 28 sec latency 17153.021 ms 2024-04-23T18:24:24.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:23 smithi172 ceph-mon[32456]: pgmap v121: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-04-23T18:24:24.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:23 smithi173 ceph-mon[32906]: pgmap v121: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-04-23T18:24:24.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:23 smithi037 ceph-mon[29404]: pgmap v121: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.6 MiB/s wr, 79 op/s 2024-04-23T18:24:24.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.40 MB/sec execute 29 sec latency 18153.163 ms 2024-04-23T18:24:25.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.39 MB/sec execute 30 sec latency 19153.260 ms 2024-04-23T18:24:26.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:25 smithi172 ceph-mon[32456]: pgmap v122: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 892 KiB/s wr, 52 op/s 2024-04-23T18:24:26.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:25 smithi173 ceph-mon[32906]: pgmap v122: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 892 KiB/s wr, 52 op/s 2024-04-23T18:24:26.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:25 smithi037 ceph-mon[29404]: pgmap v122: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 892 KiB/s wr, 52 op/s 2024-04-23T18:24:26.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241473 0.38 MB/sec execute 31 sec latency 20153.347 ms 2024-04-23T18:24:27.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:27.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2951804442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c10c570 msgr2=0x7f1c6c10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2951804442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c10c570 0x7f1c6c10e9e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009ca0 tx=0x7f1c6402a950 comp rx=0 tx=0).stop 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2951804442 shutdown_connections 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2951804442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c10c570 0x7f1c6c10e9e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2951804442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 0x7f1c6c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2951804442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c6c103230 0x7f1c6c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2951804442 >> 172.21.15.37:0/2951804442 conn(0x7f1c6c077fb0 msgr2=0x7f1c6c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.204+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2951804442 shutdown_connections 2024-04-23T18:24:27.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.205+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2951804442 wait complete. 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.205+0000 7f1c70ae8640 1 Processor -- start 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.205+0000 7f1c70ae8640 1 -- start start 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 0x7f1c6c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 0x7f1c6c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c6c10c570 0x7f1c6c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1121b0 con 0x7f1c6c103230 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c112030 con 0x7f1c6c10c570 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c112330 con 0x7f1c6c103b50 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 0x7f1c6c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 0x7f1c6c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37912/0 (socket says 172.21.15.37:37912) 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 -- 172.21.15.37:0/2329740769 learned_addr learned my addr 172.21.15.37:0/2329740769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6ad9d640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c6c10c570 0x7f1c6c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c69d9b640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 0x7f1c6c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 -- 172.21.15.37:0/2329740769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 msgr2=0x7f1c6c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 0x7f1c6c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 -- 172.21.15.37:0/2329740769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c6c10c570 msgr2=0x7f1c6c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c6c10c570 0x7f1c6c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 -- 172.21.15.37:0/2329740769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19d610 con 0x7f1c6c103230 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c69d9b640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 0x7f1c6c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c6a59c640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 0x7f1c6c198670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c920 tx=0x7f1c5400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54007c70 con 0x7f1c6c103230 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c54007e10 con 0x7f1c6c103230 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.206+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19d900 con 0x7f1c6c103230 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.207+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54018820 con 0x7f1c6c103230 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.207+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a5190 con 0x7f1c6c103230 2024-04-23T18:24:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.207+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c0691c0 con 0x7f1c6c103230 2024-04-23T18:24:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.209+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c5400f450 con 0x7f1c6c103230 2024-04-23T18:24:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.210+0000 7f1c637fe640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1c580a3d20 0x7f1c580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.210+0000 7f1c69d9b640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1c580a3d20 0x7f1c580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.210+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f1c540d0c40 con 0x7f1c6c103230 2024-04-23T18:24:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.210+0000 7f1c69d9b640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1c580a3d20 0x7f1c580a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c005d90 tx=0x7f1c5c005d20 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.214+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c54014030 con 0x7f1c6c103230 2024-04-23T18:24:27.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.437+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c6c199970 con 0x7f1c6c103230 2024-04-23T18:24:27.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.438+0000 7f1c637fe640 1 -- 172.21.15.37:0/2329740769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11881 (secure 0 0 0) 0x7f1c5409d300 con 0x7f1c6c103230 2024-04-23T18:24:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:24:27.431 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:24:08.541678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3035435067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3035435067},{"type":"v1","addr":"172.21.15.173:6837","nonce":3035435067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24484,24509,24479]},"id":1}]} 2024-04-23T18:24:27.431 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1c580a3d20 msgr2=0x7f1c580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1c580a3d20 0x7f1c580a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c005d90 tx=0x7f1c5c005d20 comp rx=0 tx=0).stop 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 msgr2=0x7f1c6c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 0x7f1c6c198670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c920 tx=0x7f1c5400cde0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 shutdown_connections 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1c580a3d20 0x7f1c580a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c6c10c570 0x7f1c6c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c6c103b50 0x7f1c6c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 --2- 172.21.15.37:0/2329740769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c6c103230 0x7f1c6c198670 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 >> 172.21.15.37:0/2329740769 conn(0x7f1c6c077fb0 msgr2=0x7f1c6c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 shutdown_connections 2024-04-23T18:24:27.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.440+0000 7f1c70ae8640 1 -- 172.21.15.37:0/2329740769 wait complete. 2024-04-23T18:24:27.444 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id c72b2d62 --release --await' 2024-04-23T18:24:27.444 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id c72b2d62 --release --await 2024-04-23T18:24:27.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 -- 172.21.15.37:0/3905412765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074c40 msgr2=0x7fe1d010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3905412765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074c40 0x7fe1d010e280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009900 tx=0x7fe1c402a480 comp rx=0 tx=0).stop 2024-04-23T18:24:27.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 -- 172.21.15.37:0/3905412765 shutdown_connections 2024-04-23T18:24:27.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3905412765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d010e7c0 0x7fe1d0110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3905412765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074c40 0x7fe1d010e280 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3905412765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d0074320 0x7fe1d0074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.560+0000 7fe1d7144640 1 -- 172.21.15.37:0/3905412765 >> 172.21.15.37:0/3905412765 conn(0x7fe1d00fd990 msgr2=0x7fe1d00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.561+0000 7fe1d7144640 1 -- 172.21.15.37:0/3905412765 shutdown_connections 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.561+0000 7fe1d7144640 1 -- 172.21.15.37:0/3905412765 wait complete. 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 Processor -- start 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 -- start start 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 0x7fe1d010e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d0074c40 0x7fe1d010b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d010e7c0 0x7fe1d010b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01143d0 con 0x7fe1d0074320 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0114250 con 0x7fe1d0074c40 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d7144640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d0114550 con 0x7fe1d010e7c0 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 0x7fe1d010e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 0x7fe1d010e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37936/0 (socket says 172.21.15.37:37936) 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/3248574610 learned_addr learned my addr 172.21.15.37:0/3248574610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d010e7c0 0x7fe1d010b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.562+0000 7fe1cffff640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d0074c40 0x7fe1d010b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/3248574610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d010e7c0 msgr2=0x7fe1d010b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d010e7c0 0x7fe1d010b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/3248574610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d0074c40 msgr2=0x7fe1d010b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d0074c40 0x7fe1d010b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/3248574610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d01ab110 con 0x7fe1d0074320 2024-04-23T18:24:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 0x7fe1d010e1b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c920 tx=0x7fe1c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1cdffb640 1 -- 172.21.15.37:0/3248574610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0007c70 con 0x7fe1d0074320 2024-04-23T18:24:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1cdffb640 1 -- 172.21.15.37:0/3248574610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0007e10 con 0x7fe1d0074320 2024-04-23T18:24:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d01ab2e0 con 0x7fe1d0074320 2024-04-23T18:24:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1cdffb640 1 -- 172.21.15.37:0/3248574610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0018820 con 0x7fe1d0074320 2024-04-23T18:24:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.563+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d01ab7a0 con 0x7fe1d0074320 2024-04-23T18:24:27.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.565+0000 7fe1cdffb640 1 -- 172.21.15.37:0/3248574610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1c000f450 con 0x7fe1d0074320 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 Processor -- start 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 -- start start 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fda870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1cdffb640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe1b80a3d20 0x7fe1b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e552f3f6d0 con 0x55e552fda490 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e552edc1a0 con 0x55e552fdadb0 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e552f3b690 con 0x55e552fe5480 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fda870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37944/0 (socket says 172.21.15.37:37944) 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 -- 172.21.15.37:0/2065769889 learned_addr learned my addr 172.21.15.37:0/2065769889 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe1b80a3d20 0x7fe1b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1cffff640 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1cdffb640 1 -- 172.21.15.37:0/3248574610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe1c00d0880 con 0x7fe1d0074320 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 -- 172.21.15.37:0/2065769889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 msgr2=0x55e552fe7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 -- 172.21.15.37:0/2065769889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 msgr2=0x55e552fe4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d56e1640 1 -- 172.21.15.37:0/2065769889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e552ee65b0 con 0x55e552fda490 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.566+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fda870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8012cb0 tx=0x7fe1c8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40946e1d1f41f439 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe1b80a3d20 0x7fe1b80a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009fd0 tx=0x7fe1c8017000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1b6ffd640 1 -- 172.21.15.37:0/2065769889 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8023030 con 0x55e552fda490 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1b6ffd640 1 -- 172.21.15.37:0/2065769889 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c8012e60 con 0x55e552fda490 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1b6ffd640 1 -- 172.21.15.37:0/2065769889 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c800dbb0 con 0x55e552fda490 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2065769889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 msgr2=0x55e552fda870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fda870 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8012cb0 tx=0x7fe1c8010740 comp rx=0 tx=0).stop 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2065769889 shutdown_connections 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.567+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2065769889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fda870 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2065769889 >> 172.21.15.37:0/2065769889 conn(0x55e552fd67b0 msgr2=0x55e552fd6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2065769889 shutdown_connections 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2065769889 wait complete. 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- start start 2024-04-23T18:24:27.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 Processor -- start 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fdadb0 0x55e552fe27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fe5480 0x55e552fe2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e552fe82b0 con 0x55e552fda490 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e552ed9b20 con 0x55e552fe5480 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e552fe8430 con 0x55e552fdadb0 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d9e69740 10 client.0 _ll_register_callbacks cb 0x55e552ed9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.568+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37958/0 (socket says 172.21.15.37:37958) 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/348757350 learned_addr learned my addr 172.21.15.37:0/348757350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fe5480 0x55e552fe2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/348757350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fdadb0 msgr2=0x55e552fe27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1cffff640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fdadb0 0x55e552fe27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fdadb0 0x55e552fe27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fe5480 msgr2=0x55e552fe2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fe5480 0x55e552fe2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/348757350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e552c51920 con 0x55e552fda490 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fe5480 0x55e552fe2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe2270 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0098c70 tx=0x7fe1c000a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e741edf515d6e72b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1affff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1affff640 1 -- 172.21.15.37:0/348757350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00a0bc0 con 0x55e552fda490 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1affff640 1 -- 172.21.15.37:0/348757350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe1c00ce240 con 0x55e552fda490 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c8012e60 con 0x55e552fda490 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1affff640 1 -- 172.21.15.37:0/348757350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1c00a5480 con 0x55e552fda490 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d9e69740 10 client.16020 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e552d1dd30 con 0x55e552fda490 2024-04-23T18:24:27.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.569+0000 7fe1affff640 1 -- 172.21.15.37:0/348757350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c8012e60 con 0x55e552fda490 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1affff640 1 -- 172.21.15.37:0/348757350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe1c00a4ff0 con 0x55e552fda490 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1d9e69740 10 client.16020 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1d9e69740 10 client.16020 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1d9e69740 20 client.16020 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e552ff56f0 0x55e552ff7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1d9e69740 4 client.16020 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e552efd500 con 0x55e552ff56f0 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1cffff640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e552ff56f0 0x55e552ff7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1affff640 10 client.16020 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.570+0000 7fe1cffff640 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e552ff56f0 0x55e552ff7b60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e37732ddbf8986a server_cookie=6edb56b3c8c6e5fd in_seq=0 out_seq=0 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1affff640 1 -- 172.21.15.37:0/348757350 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e552efd500 con 0x55e552ff56f0 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1affff640 10 client.16020 handle_command_reply: tid=0 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 client.16020 shutdown 2024-04-23T18:24:27.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 msgr2=0x55e552fe2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe2270 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0098c70 tx=0x7fe1c000a5e0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 shutdown_connections 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e552ff56f0 0x55e552ff7b60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fe5480 0x55e552fe2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1affff640 0 client.16020 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fdadb0 0x55e552fe27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/348757350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe2270 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.571+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 >> 172.21.15.37:0/348757350 conn(0x55e552fd67b0 msgr2=0x55e552fd59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.572+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 shutdown_connections 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.572+0000 7fe1d9e69740 1 -- 172.21.15.37:0/348757350 wait complete. 2024-04-23T18:24:27.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.572+0000 7fe1d9e69740 20 client.16020 trim_cache size 0 max 16384 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.589+0000 7fe1d9e69740 1 Processor -- start 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.589+0000 7fe1d9e69740 1 -- start start 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552ff1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fd6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01b6980 con 0x55e552fda490 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01b41f0 con 0x55e552fdadb0 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01b2040 con 0x55e552fe5480 2024-04-23T18:24:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552ff1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552ff1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37966/0 (socket says 172.21.15.37:37966) 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/2274251008 learned_addr learned my addr 172.21.15.37:0/2274251008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1cffff640 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fd6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/2274251008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 msgr2=0x55e552fe2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/2274251008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 msgr2=0x55e552fd6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fd6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/2274251008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e552fe07c0 con 0x55e552fda490 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1cffff640 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fd6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552ff1170 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c8f0 tx=0x7fe1c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcab0b1a102293e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1b67fc640 1 -- 172.21.15.37:0/2274251008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0098e80 con 0x55e552fda490 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1b67fc640 1 -- 172.21.15.37:0/2274251008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c00870b0 con 0x55e552fda490 2024-04-23T18:24:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.590+0000 7fe1b67fc640 1 -- 172.21.15.37:0/2274251008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0098560 con 0x55e552fda490 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2274251008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 msgr2=0x55e552ff1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552ff1170 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c8f0 tx=0x7fe1c0000bc0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2274251008 shutdown_connections 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fd6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/2274251008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552ff1170 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2274251008 >> 172.21.15.37:0/2274251008 conn(0x55e552fd67b0 msgr2=0x55e552fe12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2274251008 shutdown_connections 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 -- 172.21.15.37:0/2274251008 wait complete. 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 -- start start 2024-04-23T18:24:27.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.591+0000 7fe1d9e69740 1 Processor -- start 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01b26b0 con 0x55e552fda490 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01b3b20 con 0x55e552fdadb0 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d01b2d80 con 0x55e552fe5480 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d9e69740 10 client.0 _ll_register_callbacks cb 0x7fe1c009c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37980/0 (socket says 172.21.15.37:37980) 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/779497563 learned_addr learned my addr 172.21.15.37:0/779497563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d56e1640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1cffff640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/779497563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 msgr2=0x55e552fe6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 msgr2=0x55e552fe68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 -- 172.21.15.37:0/779497563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e552feba70 con 0x55e552fda490 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1d4ee0640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe6390 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1c009c7a0 tx=0x7fe1c00a5220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1f2dbad36d4e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.592+0000 7fe1affff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1affff640 1 -- 172.21.15.37:0/779497563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00a9070 con 0x55e552fda490 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1affff640 1 -- 172.21.15.37:0/779497563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe1c00aeb60 con 0x55e552fda490 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c00870b0 con 0x55e552fda490 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1affff640 1 -- 172.21.15.37:0/779497563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1c009db40 con 0x55e552fda490 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 10 client.16026 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e55305c0d0 con 0x55e552fda490 2024-04-23T18:24:27.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1affff640 1 -- 172.21.15.37:0/779497563 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c00870b0 con 0x55e552fda490 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1affff640 1 -- 172.21.15.37:0/779497563 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe1c00ac070 con 0x55e552fda490 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 10 client.16026 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 10 client.16026 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 20 client.16026 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e55305c5f0 0x55e55305e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 4 client.16026 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c72b2d62", "release": true, "await": true}] 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.593+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c72b2d62", "release": true, "await": true}) -- 0x55e552f4d0f0 con 0x55e55305c5f0 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.594+0000 7fe1cffff640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e55305c5f0 0x55e55305e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.594+0000 7fe1affff640 10 client.16026 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.594+0000 7fe1cffff640 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e55305c5f0 0x55e55305e9e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82178d5da8539ec server_cookie=16683303cc3a23cb in_seq=0 out_seq=0 2024-04-23T18:24:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1affff640 1 -- 172.21.15.37:0/779497563 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e552f4d0f0 con 0x55e55305c5f0 2024-04-23T18:24:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1affff640 10 client.16026 handle_command_reply: tid=0 2024-04-23T18:24:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 client.16026 shutdown 2024-04-23T18:24:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 msgr2=0x55e552fe6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe6390 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1c009c7a0 tx=0x7fe1c00a5220 comp rx=0 tx=0).stop 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 shutdown_connections 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e55305c5f0 0x55e55305e9e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e552fe5480 0x55e552fe6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e552fdadb0 0x55e552fe68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 --2- 172.21.15.37:0/779497563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e552fda490 0x55e552fe6390 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1affff640 0 client.16026 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 >> 172.21.15.37:0/779497563 conn(0x55e552fd67b0 msgr2=0x55e552fdc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 shutdown_connections 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 1 -- 172.21.15.37:0/779497563 wait complete. 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.599+0000 7fe1d9e69740 20 client.16026 trim_cache size 0 max 16384 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:24:27.593 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 16, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "c72b2d62": { 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 16, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 80.3, 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:24:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:24:27.595 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe1b80a3d20 msgr2=0x7fe1b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe1b80a3d20 0x7fe1b80a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009fd0 tx=0x7fe1c8017000 comp rx=0 tx=0).stop 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 msgr2=0x7fe1d010e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 0x7fe1d010e1b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c920 tx=0x7fe1c000cde0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 shutdown_connections 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe1b80a3d20 0x7fe1b80a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d010e7c0 0x7fe1d010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d0074c40 0x7fe1d010b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 --2- 172.21.15.37:0/3248574610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d0074320 0x7fe1d010e1b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 >> 172.21.15.37:0/3248574610 conn(0x7fe1d00fd990 msgr2=0x7fe1d00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 shutdown_connections 2024-04-23T18:24:27.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:24:27.600+0000 7fe1d7144640 1 -- 172.21.15.37:0/3248574610 wait complete. 2024-04-23T18:24:27.603 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c72b2d62', seconds elapsed: 21.5 2024-04-23T18:24:27.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241629 0.37 MB/sec execute 32 sec latency 20936.600 ms 2024-04-23T18:24:27.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:27 smithi037 ceph-mon[29404]: pgmap v123: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 908 KiB/s wr, 58 op/s 2024-04-23T18:24:27.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:27 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2329740769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:27 smithi172 ceph-mon[32456]: pgmap v123: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 908 KiB/s wr, 58 op/s 2024-04-23T18:24:28.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:27 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2329740769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:27 smithi173 ceph-mon[32906]: pgmap v123: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 908 KiB/s wr, 58 op/s 2024-04-23T18:24:28.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:27 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2329740769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 243707 0.60 MB/sec execute 33 sec latency 15.383 ms 2024-04-23T18:24:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 -- 172.21.15.37:0/2354840972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6190103b00 msgr2=0x7f619010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2354840972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6190103b00 0x7f619010bfe0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6180009900 tx=0x7f618002a510 comp rx=0 tx=0).stop 2024-04-23T18:24:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 -- 172.21.15.37:0/2354840972 shutdown_connections 2024-04-23T18:24:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2354840972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f619010c520 0x7f619010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2354840972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6190103b00 0x7f619010bfe0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2354840972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61901031e0 0x7f61901035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 -- 172.21.15.37:0/2354840972 >> 172.21.15.37:0/2354840972 conn(0x7f6190077fd0 msgr2=0x7f61900783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 -- 172.21.15.37:0/2354840972 shutdown_connections 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.800+0000 7f6197ff9640 1 -- 172.21.15.37:0/2354840972 wait complete. 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 Processor -- start 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 -- start start 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 0x7f6190198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6190103b00 0x7f6190198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f619010c520 0x7f619019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61901121b0 con 0x7f61901031e0 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112030 con 0x7f6190103b00 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6197ff9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112330 con 0x7f619010c520 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6195d95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 0x7f6190198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6195d95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 0x7f6190198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37992/0 (socket says 172.21.15.37:37992) 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6195d95640 1 -- 172.21.15.37:0/2216300594 learned_addr learned my addr 172.21.15.37:0/2216300594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6196596640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f619010c520 0x7f619019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6195594640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6190103b00 0x7f6190198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6195d95640 1 -- 172.21.15.37:0/2216300594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f619010c520 msgr2=0x7f619019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.801+0000 7f6195d95640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f619010c520 0x7f619019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6195d95640 1 -- 172.21.15.37:0/2216300594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6190103b00 msgr2=0x7f6190198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6195d95640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6190103b00 0x7f6190198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6195d95640 1 -- 172.21.15.37:0/2216300594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619019d6b0 con 0x7f61901031e0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6195d95640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 0x7f6190198710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f618c00d950 tx=0x7f618c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6186ffd640 1 -- 172.21.15.37:0/2216300594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c017030 con 0x7f61901031e0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6186ffd640 1 -- 172.21.15.37:0/2216300594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c004020 con 0x7f61901031e0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619019d9a0 con 0x7f61901031e0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6186ffd640 1 -- 172.21.15.37:0/2216300594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c0047a0 con 0x7f61901031e0 2024-04-23T18:24:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.802+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901a5230 con 0x7f61901031e0 2024-04-23T18:24:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.804+0000 7f6186ffd640 1 -- 172.21.15.37:0/2216300594 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f618c009a30 con 0x7f61901031e0 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 Processor -- start 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6186ffd640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f617c0a3e40 0x7f617c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6186ffd640 1 -- 172.21.15.37:0/2216300594 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f618c0d0130 con 0x7f61901031e0 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6196596640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f617c0a3e40 0x7f617c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 -- start start 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7e8ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7e9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aff7df3970 con 0x55aff7e99800 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55aff7d904a0 con 0x55aff7e8e620 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f619ade0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aff7def930 con 0x55aff7e8ef40 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6196596640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f617c0a3e40 0x7f617c0a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6188005fd0 tx=0x7f6188009500 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6196596640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7e9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6195d95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7e8ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6195594640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6196596640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7e9bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38002/0 (socket says 172.21.15.37:38002) 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.805+0000 7f6196596640 1 -- 172.21.15.37:0/3946543697 learned_addr learned my addr 172.21.15.37:0/3946543697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6195d95640 1 -- 172.21.15.37:0/3946543697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 msgr2=0x55aff7e992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6195d95640 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6195d95640 1 -- 172.21.15.37:0/3946543697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 msgr2=0x55aff7e9bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6195d95640 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7e9bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6195d95640 1 -- 172.21.15.37:0/3946543697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aff7b06210 con 0x55aff7e8e620 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6196596640 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7e9bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f6195d95640 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7e8ea00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cd870 tx=0x7f618c09bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ca064b0b9f945a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f617bfff640 1 -- 172.21.15.37:0/3946543697 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c0b5070 con 0x55aff7e8e620 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f617bfff640 1 -- 172.21.15.37:0/3946543697 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c0a0470 con 0x55aff7e8e620 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.806+0000 7f617bfff640 1 -- 172.21.15.37:0/3946543697 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c0cdaf0 con 0x55aff7e8e620 2024-04-23T18:24:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 -- 172.21.15.37:0/3946543697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 msgr2=0x55aff7e8ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7e8ea00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cd870 tx=0x7f618c09bd30 comp rx=0 tx=0).stop 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 -- 172.21.15.37:0/3946543697 shutdown_connections 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7e9bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 --2- 172.21.15.37:0/3946543697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7e8ea00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 -- 172.21.15.37:0/3946543697 >> 172.21.15.37:0/3946543697 conn(0x55aff7e8ab30 msgr2=0x55aff7e8af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 -- 172.21.15.37:0/3946543697 shutdown_connections 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 -- 172.21.15.37:0/3946543697 wait complete. 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 -- start start 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.807+0000 7f619ade0740 1 Processor -- start 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 0x55aff7e99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e96480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e99800 0x55aff7e96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aff7e9c430 con 0x55aff7e8e620 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55aff7d8e0b0 con 0x55aff7e99800 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aff7e9c5b0 con 0x55aff7e8ef40 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f619ade0740 10 client.0 _ll_register_callbacks cb 0x55aff7d7bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 0x55aff7e99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 0x55aff7e99190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38010/0 (socket says 172.21.15.37:38010) 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 -- 172.21.15.37:0/3042563517 learned_addr learned my addr 172.21.15.37:0/3042563517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195594640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e96480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6196596640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e99800 0x55aff7e96a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 -- 172.21.15.37:0/3042563517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 msgr2=0x55aff7e96480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e96480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 -- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e99800 msgr2=0x55aff7e96a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e99800 0x55aff7e96a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 -- 172.21.15.37:0/3042563517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aff7bd2620 con 0x55aff7e8e620 2024-04-23T18:24:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f6195d95640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 0x55aff7e99190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cd8d0 tx=0x7f618c09c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd291f893942377 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.808+0000 7f61797fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f61797fa640 1 -- 172.21.15.37:0/3042563517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c0b5040 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f61797fa640 1 -- 172.21.15.37:0/3042563517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f618c0cdaf0 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618c0a0470 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f61797fa640 1 -- 172.21.15.37:0/3042563517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f618c009d20 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 10 client.16038 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aff7ea8380 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f61797fa640 1 -- 172.21.15.37:0/3042563517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c0a0470 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f61797fa640 1 -- 172.21.15.37:0/3042563517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f618c0b0b60 con 0x55aff7e8e620 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 10 client.16038 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 20 client.16038 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7ea96d0 0x55aff7eabac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 4 client.16038 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.809+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aff7e01390 con 0x55aff7ea96d0 2024-04-23T18:24:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.810+0000 7f6195594640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7ea96d0 0x55aff7eabac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.810+0000 7f6195594640 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7ea96d0 0x55aff7eabac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c132f760ed553a0e server_cookie=b9c012497ae3b178 in_seq=0 out_seq=0 2024-04-23T18:24:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.810+0000 7f61797fa640 10 client.16038 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f61797fa640 1 -- 172.21.15.37:0/3042563517 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aff7e01390 con 0x55aff7ea96d0 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f61797fa640 10 client.16038 handle_command_reply: tid=0 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 client.16038 shutdown 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 msgr2=0x55aff7e99190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 0x55aff7e99190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f618c0cd8d0 tx=0x7f618c09c180 comp rx=0 tx=0).stop 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 shutdown_connections 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7ea96d0 0x55aff7eabac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e99800 0x55aff7e96a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8ef40 0x55aff7e96480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 --2- 172.21.15.37:0/3042563517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8e620 0x55aff7e99190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f61797fa640 0 client.16038 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 >> 172.21.15.37:0/3042563517 conn(0x55aff7e8ab30 msgr2=0x55aff7e908b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 shutdown_connections 2024-04-23T18:24:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 1 -- 172.21.15.37:0/3042563517 wait complete. 2024-04-23T18:24:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.811+0000 7f619ade0740 20 client.16038 trim_cache size 0 max 16384 2024-04-23T18:24:29.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 245463 0.92 MB/sec execute 34 sec latency 10.856 ms 2024-04-23T18:24:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 Processor -- start 2024-04-23T18:24:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 -- start start 2024-04-23T18:24:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8e620 0x55aff7ea4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 0x55aff7e98cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7eaeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61901b0490 con 0x55aff7e99800 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61901add00 con 0x55aff7e8ef40 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f619ade0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61901abb50 con 0x55aff7e8e620 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.826+0000 7f6196596640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7eaeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6195594640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 0x55aff7e98cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6195d95640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8e620 0x55aff7ea4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7eaeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38012/0 (socket says 172.21.15.37:38012) 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6195d95640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8e620 0x55aff7ea4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:52524/0 (socket says 172.21.15.37:52524) 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6195594640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 0x55aff7e98cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:48312/0 (socket says 172.21.15.37:48312) 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6195d95640 1 -- 172.21.15.37:0/2157881451 learned_addr learned my addr 172.21.15.37:0/2157881451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 -- 172.21.15.37:0/2157881451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8e620 msgr2=0x55aff7ea4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8e620 0x55aff7ea4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 -- 172.21.15.37:0/2157881451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 msgr2=0x55aff7e98cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 0x55aff7e98cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 -- 172.21.15.37:0/2157881451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aff7e949f0 con 0x55aff7e99800 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6195594640 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 0x55aff7e98cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f6196596640 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7eaeae0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6188005bd0 tx=0x7f6188014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3542b1b48637759b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f617b7fe640 1 -- 172.21.15.37:0/2157881451 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618802d070 con 0x55aff7e99800 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f617b7fe640 1 -- 172.21.15.37:0/2157881451 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880288c0 con 0x55aff7e99800 2024-04-23T18:24:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.827+0000 7f617b7fe640 1 -- 172.21.15.37:0/2157881451 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618801f940 con 0x55aff7e99800 2024-04-23T18:24:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 -- 172.21.15.37:0/2157881451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 msgr2=0x55aff7eaeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7eaeae0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6188005bd0 tx=0x7f6188014000 comp rx=0 tx=0).stop 2024-04-23T18:24:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 -- 172.21.15.37:0/2157881451 shutdown_connections 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e99800 0x55aff7eaeae0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8ef40 0x55aff7e98cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 --2- 172.21.15.37:0/2157881451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e8e620 0x55aff7ea4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 -- 172.21.15.37:0/2157881451 >> 172.21.15.37:0/2157881451 conn(0x55aff7e8ab30 msgr2=0x55aff7ea4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 -- 172.21.15.37:0/2157881451 shutdown_connections 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 -- 172.21.15.37:0/2157881451 wait complete. 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.828+0000 7f619ade0740 1 -- start start 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 Processor -- start 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7eaedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 0x55aff7eac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e99800 0x55aff7eac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61901ac1c0 con 0x55aff7e8ef40 2024-04-23T18:24:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61901ad630 con 0x55aff7e8e620 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61901ac890 con 0x55aff7e99800 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f619ade0740 10 client.0 _ll_register_callbacks cb 0x7f618c09baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f6195594640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 0x55aff7eac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f6196596640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e99800 0x55aff7eac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f6195594640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 0x55aff7eac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38022/0 (socket says 172.21.15.37:38022) 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f6195d95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7eaedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f6195594640 1 -- 172.21.15.37:0/1042576660 learned_addr learned my addr 172.21.15.37:0/1042576660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.829+0000 7f6195594640 1 -- 172.21.15.37:0/1042576660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e99800 msgr2=0x55aff7eac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f6195594640 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e99800 0x55aff7eac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f6195594640 1 -- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 msgr2=0x55aff7eaedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f6195594640 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7eaedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f6195594640 1 -- 172.21.15.37:0/1042576660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aff7eacec0 con 0x55aff7e8ef40 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f6195d95640 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7eaedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f6195594640 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 0x55aff7eac180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6180009900 tx=0x7f618002acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d582e5c1b39292e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f61797fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f61797fa640 1 -- 172.21.15.37:0/1042576660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618004a070 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f61797fa640 1 -- 172.21.15.37:0/1042576660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f61800042c0 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61880288c0 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f61797fa640 1 -- 172.21.15.37:0/1042576660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6180056050 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f619ade0740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aff7f0f890 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.830+0000 7f61797fa640 1 -- 172.21.15.37:0/1042576660 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880288c0 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f61797fa640 1 -- 172.21.15.37:0/1042576660 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6180046630 con 0x55aff7e8ef40 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f619ade0740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f619ade0740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f619ade0740 20 client.16044 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f619ade0740 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7f100d0 0x55aff7f124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f619ade0740 4 client.16044 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:24:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aff7dfb200 con 0x55aff7f100d0 2024-04-23T18:24:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f6195d95640 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7f100d0 0x55aff7f124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f61797fa640 10 client.16044 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.831+0000 7f6195d95640 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7f100d0 0x55aff7f124f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee46ada00b4ad6cc server_cookie=b1dcaed657451123 in_seq=0 out_seq=0 2024-04-23T18:24:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f61797fa640 1 -- 172.21.15.37:0/1042576660 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aff7dfb200 con 0x55aff7f100d0 2024-04-23T18:24:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f61797fa640 10 client.16044 handle_command_reply: tid=0 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 client.16044 shutdown 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 msgr2=0x55aff7eac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 0x55aff7eac180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6180009900 tx=0x7f618002acc0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 shutdown_connections 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55aff7f100d0 0x55aff7f124f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aff7e99800 0x55aff7eac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aff7e8ef40 0x55aff7eac180 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f61797fa640 0 client.16044 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 --2- 172.21.15.37:0/1042576660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aff7e8e620 0x55aff7eaedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 >> 172.21.15.37:0/1042576660 conn(0x55aff7e8ab30 msgr2=0x55aff7e90860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 shutdown_connections 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 1 -- 172.21.15.37:0/1042576660 wait complete. 2024-04-23T18:24:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.868+0000 7f619ade0740 20 client.16044 trim_cache size 0 max 16384 2024-04-23T18:24:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:24:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:24:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:24:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f617c0a3e40 msgr2=0x7f617c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f617c0a3e40 0x7f617c0a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6188005fd0 tx=0x7f6188009500 comp rx=0 tx=0).stop 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 msgr2=0x7f6190198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 0x7f6190198710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f618c00d950 tx=0x7f618c00de10 comp rx=0 tx=0).stop 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 shutdown_connections 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f617c0a3e40 0x7f617c0a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f619010c520 0x7f619019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6190103b00 0x7f6190198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 --2- 172.21.15.37:0/2216300594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61901031e0 0x7f6190198710 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 >> 172.21.15.37:0/2216300594 conn(0x7f6190077fd0 msgr2=0x7f61900fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 shutdown_connections 2024-04-23T18:24:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:29.869+0000 7f6197ff9640 1 -- 172.21.15.37:0/2216300594 wait complete. 2024-04-23T18:24:30.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:29 smithi172 ceph-mon[32456]: pgmap v124: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 127 KiB/s wr, 20 op/s 2024-04-23T18:24:30.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:29 smithi173 ceph-mon[32906]: pgmap v124: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 127 KiB/s wr, 20 op/s 2024-04-23T18:24:30.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:29 smithi037 ceph-mon[29404]: pgmap v124: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 127 KiB/s wr, 20 op/s 2024-04-23T18:24:30.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 248328 1.39 MB/sec execute 35 sec latency 15.527 ms 2024-04-23T18:24:31.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 249888 1.54 MB/sec execute 36 sec latency 15.853 ms 2024-04-23T18:24:32.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:31 smithi172 ceph-mon[32456]: pgmap v125: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 127 KiB/s wr, 20 op/s 2024-04-23T18:24:32.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:31 smithi173 ceph-mon[32906]: pgmap v125: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 127 KiB/s wr, 20 op/s 2024-04-23T18:24:32.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:31 smithi037 ceph-mon[29404]: pgmap v125: 97 pgs: 97 active+clean; 124 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 127 KiB/s wr, 20 op/s 2024-04-23T18:24:32.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 252251 1.87 MB/sec execute 37 sec latency 12.199 ms 2024-04-23T18:24:33.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 254304 2.01 MB/sec execute 38 sec latency 18.250 ms 2024-04-23T18:24:34.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:33 smithi172 ceph-mon[32456]: pgmap v126: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 732 KiB/s wr, 118 op/s 2024-04-23T18:24:34.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:33 smithi173 ceph-mon[32906]: pgmap v126: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 732 KiB/s wr, 118 op/s 2024-04-23T18:24:34.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:33 smithi037 ceph-mon[29404]: pgmap v126: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 732 KiB/s wr, 118 op/s 2024-04-23T18:24:34.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 256013 2.27 MB/sec execute 39 sec latency 10.346 ms 2024-04-23T18:24:35.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 258694 2.57 MB/sec execute 40 sec latency 16.553 ms 2024-04-23T18:24:36.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:35 smithi172 ceph-mon[32456]: pgmap v127: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 620 KiB/s wr, 103 op/s 2024-04-23T18:24:36.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:35 smithi173 ceph-mon[32906]: pgmap v127: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 620 KiB/s wr, 103 op/s 2024-04-23T18:24:36.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:35 smithi037 ceph-mon[29404]: pgmap v127: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 620 KiB/s wr, 103 op/s 2024-04-23T18:24:36.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 260111 2.67 MB/sec execute 41 sec latency 14.321 ms 2024-04-23T18:24:37.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 262756 3.01 MB/sec execute 42 sec latency 20.183 ms 2024-04-23T18:24:38.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:37 smithi172 ceph-mon[32456]: pgmap v128: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-23T18:24:38.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:37 smithi173 ceph-mon[32906]: pgmap v128: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-23T18:24:38.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:37 smithi037 ceph-mon[29404]: pgmap v128: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 207 op/s 2024-04-23T18:24:38.816 INFO:tasks.workunit.client.0.smithi037.stdout: 1 264373 3.07 MB/sec execute 43 sec latency 14.781 ms 2024-04-23T18:24:39.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 266459 3.31 MB/sec execute 44 sec latency 11.688 ms 2024-04-23T18:24:40.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:39 smithi172 ceph-mon[32456]: pgmap v129: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-23T18:24:40.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:39 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:40.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:39 smithi173 ceph-mon[32906]: pgmap v129: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-23T18:24:40.105 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:39 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:40.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:39 smithi037 ceph-mon[29404]: pgmap v129: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-23T18:24:40.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:39 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:40.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 268490 3.40 MB/sec execute 45 sec latency 22.935 ms 2024-04-23T18:24:41.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:41.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:41.087 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:24:41.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:41.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:41.104 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:24:41.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:41.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:41.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:24:41.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 270198 3.59 MB/sec execute 46 sec latency 15.191 ms 2024-04-23T18:24:42.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:41 smithi037 ceph-mon[29404]: pgmap v130: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-23T18:24:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:41 smithi172 ceph-mon[32456]: pgmap v130: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-23T18:24:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:41 smithi173 ceph-mon[32906]: pgmap v130: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-23T18:24:42.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 272875 3.82 MB/sec execute 47 sec latency 17.269 ms 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/1088876001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a481031d0 msgr2=0x7f0a481035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/1088876001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a481031d0 0x7f0a481035b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009900 tx=0x7f0a4402a480 comp rx=0 tx=0).stop 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/1088876001 shutdown_connections 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/1088876001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a4810c5a0 0x7f0a4810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/1088876001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a48103af0 0x7f0a4810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/1088876001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a481031d0 0x7f0a481035b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.960+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/1088876001 >> 172.21.15.37:0/1088876001 conn(0x7f0a48077fa0 msgr2=0x7f0a480783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/1088876001 shutdown_connections 2024-04-23T18:24:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/1088876001 wait complete. 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 Processor -- start 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 -- start start 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a481031d0 0x7f0a481a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 0x7f0a481a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a4810c5a0 0x7f0a4819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481121d0 con 0x7f0a48103af0 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112050 con 0x7f0a481031d0 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.961+0000 7f0a4f7e5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112350 con 0x7f0a4810c5a0 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 0x7f0a481a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 0x7f0a481a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51258/0 (socket says 172.21.15.37:51258) 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4d581640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a481031d0 0x7f0a481a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 -- 172.21.15.37:0/2741922299 learned_addr learned my addr 172.21.15.37:0/2741922299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a4810c5a0 0x7f0a4819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 -- 172.21.15.37:0/2741922299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a4810c5a0 msgr2=0x7f0a4819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a4810c5a0 0x7f0a4819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 -- 172.21.15.37:0/2741922299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a481031d0 msgr2=0x7f0a481a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a481031d0 0x7f0a481a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 -- 172.21.15.37:0/2741922299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4819afd0 con 0x7f0a48103af0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4d581640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a481031d0 0x7f0a481a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 0x7f0a481a0b40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800e9f0 tx=0x7f0a3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a3e7fc640 1 -- 172.21.15.37:0/2741922299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3800cd80 con 0x7f0a48103af0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a3e7fc640 1 -- 172.21.15.37:0/2741922299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a380108e0 con 0x7f0a48103af0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4819b2c0 con 0x7f0a48103af0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.962+0000 7f0a3e7fc640 1 -- 172.21.15.37:0/2741922299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38018980 con 0x7f0a48103af0 2024-04-23T18:24:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.963+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a481a7580 con 0x7f0a48103af0 2024-04-23T18:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a3e7fc640 1 -- 172.21.15.37:0/2741922299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0a38010430 con 0x7f0a48103af0 2024-04-23T18:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 Processor -- start 2024-04-23T18:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 -- start start 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a3fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557bf39a4970 con 0x557bf3a4a800 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bf39414a0 con 0x557bf3a3ff40 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a525da740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557bf39a0930 con 0x557bf3a3f620 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a3e7fc640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0a340a3e40 0x7f0a340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a4dd82640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a3fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a4cd80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a4d581640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.965+0000 7f0a4dd82640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a3fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:53670/0 (socket says 172.21.15.37:53670) 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4dd82640 1 -- 172.21.15.37:0/1728985286 learned_addr learned my addr 172.21.15.37:0/1728985286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0a340a3e40 0x7f0a340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a3e7fc640 1 -- 172.21.15.37:0/2741922299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0a380d0a80 con 0x7f0a48103af0 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4d581640 1 -- 172.21.15.37:0/1728985286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 msgr2=0x557bf3a3fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4d581640 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a3fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4d581640 1 -- 172.21.15.37:0/1728985286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 msgr2=0x557bf3a4cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4d581640 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a4cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4d581640 1 -- 172.21.15.37:0/1728985286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf36b7210 con 0x557bf3a3ff40 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a3fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4d581640 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a4a2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009230 tx=0x7f0a44005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4eb6becc470b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0a340a3e40 0x7f0a340a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a400049d0 tx=0x7f0a40004ab0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a337fe640 1 -- 172.21.15.37:0/1728985286 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44040070 con 0x557bf3a3ff40 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a337fe640 1 -- 172.21.15.37:0/1728985286 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a44031860 con 0x557bf3a3ff40 2024-04-23T18:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.966+0000 7f0a337fe640 1 -- 172.21.15.37:0/1728985286 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44035040 con 0x557bf3a3ff40 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 -- 172.21.15.37:0/1728985286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 msgr2=0x557bf3a4a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a4a2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009230 tx=0x7f0a44005d90 comp rx=0 tx=0).stop 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 -- 172.21.15.37:0/1728985286 shutdown_connections 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a4a2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 --2- 172.21.15.37:0/1728985286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a3fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 -- 172.21.15.37:0/1728985286 >> 172.21.15.37:0/1728985286 conn(0x557bf3a3bb30 msgr2=0x557bf3a3bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 -- 172.21.15.37:0/1728985286 shutdown_connections 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 -- 172.21.15.37:0/1728985286 wait complete. 2024-04-23T18:24:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 -- start start 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.967+0000 7f0a525da740 1 Processor -- start 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 0x557bf3a4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a47470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a4a800 0x557bf3a479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557bf3a4d3d0 con 0x557bf3a3f620 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bf393f0b0 con 0x557bf3a3ff40 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557bf3a4d550 con 0x557bf3a4a800 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a525da740 10 client.0 _ll_register_callbacks cb 0x557bf392cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 0x557bf3a4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 0x557bf3a4a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51290/0 (socket says 172.21.15.37:51290) 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 -- 172.21.15.37:0/1689732756 learned_addr learned my addr 172.21.15.37:0/1689732756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a47470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a4a800 0x557bf3a479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 -- 172.21.15.37:0/1689732756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a4a800 msgr2=0x557bf3a479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a4a800 0x557bf3a479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 -- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 msgr2=0x557bf3a47470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a47470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 -- 172.21.15.37:0/1689732756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf3783620 con 0x557bf3a3f620 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.968+0000 7f0a4d581640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 0x557bf3a4a180 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0a440055c0 tx=0x7f0a4402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1131cd6c74843a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a30ff9640 1 -- 172.21.15.37:0/1689732756 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4404c070 con 0x557bf3a3f620 2024-04-23T18:24:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a44031860 con 0x557bf3a3f620 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a30ff9640 1 -- 172.21.15.37:0/1689732756 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0a44003800 con 0x557bf3a3f620 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a30ff9640 1 -- 172.21.15.37:0/1689732756 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a44058050 con 0x557bf3a3f620 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 10 client.16056 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557bf3a59320 con 0x557bf3a3f620 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a30ff9640 1 -- 172.21.15.37:0/1689732756 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a44031860 con 0x557bf3a3f620 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a30ff9640 1 -- 172.21.15.37:0/1689732756 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0a4403c6a0 con 0x557bf3a3f620 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 10 client.16056 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 20 client.16056 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3a5a670 0x557bf3a5ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 4 client.16056 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.969+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf39b2390 con 0x557bf3a5a670 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.970+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3a5a670 0x557bf3a5ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.970+0000 7f0a30ff9640 10 client.16056 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.970+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3a5a670 0x557bf3a5ca60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8648a48afd350677 server_cookie=be214205fa68f7f3 in_seq=0 out_seq=0 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a30ff9640 1 -- 172.21.15.37:0/1689732756 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bf39b2390 con 0x557bf3a5a670 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a30ff9640 10 client.16056 handle_command_reply: tid=0 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 client.16056 shutdown 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 msgr2=0x557bf3a4a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 0x557bf3a4a180 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0a440055c0 tx=0x7f0a4402a480 comp rx=0 tx=0).stop 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 shutdown_connections 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3a5a670 0x557bf3a5ca60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a4a800 0x557bf3a479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3ff40 0x557bf3a47470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a30ff9640 0 client.16056 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 --2- 172.21.15.37:0/1689732756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3f620 0x557bf3a4a180 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 >> 172.21.15.37:0/1689732756 conn(0x557bf3a3bb30 msgr2=0x557bf3a41860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 shutdown_connections 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 1 -- 172.21.15.37:0/1689732756 wait complete. 2024-04-23T18:24:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.971+0000 7f0a525da740 20 client.16056 trim_cache size 0 max 16384 2024-04-23T18:24:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.987+0000 7f0a525da740 1 Processor -- start 2024-04-23T18:24:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.987+0000 7f0a525da740 1 -- start start 2024-04-23T18:24:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a525da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3f620 0x557bf3a56290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a525da740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3ff40 0x557bf3a42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a525da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a49cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a525da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481b2730 con 0x557bf3a4a800 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a525da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481affa0 con 0x557bf3a3f620 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a525da740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481addf0 con 0x557bf3a3ff40 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a49cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a49cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51296/0 (socket says 172.21.15.37:51296) 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 -- 172.21.15.37:0/4267585882 learned_addr learned my addr 172.21.15.37:0/4267585882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3ff40 0x557bf3a42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4d581640 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3f620 0x557bf3a56290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 -- 172.21.15.37:0/4267585882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3ff40 msgr2=0x557bf3a42970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3ff40 0x557bf3a42970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 -- 172.21.15.37:0/4267585882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3f620 msgr2=0x557bf3a56290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3f620 0x557bf3a56290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 -- 172.21.15.37:0/4267585882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf3a46420 con 0x557bf3a4a800 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3ff40 0x557bf3a42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a49cb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a40013270 tx=0x7f0a40005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0903afd37e33605 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.988+0000 7f0a32ffd640 1 -- 172.21.15.37:0/4267585882 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400238e0 con 0x557bf3a4a800 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a32ffd640 1 -- 172.21.15.37:0/4267585882 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40010430 con 0x557bf3a4a800 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a32ffd640 1 -- 172.21.15.37:0/4267585882 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4002b640 con 0x557bf3a4a800 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 -- 172.21.15.37:0/4267585882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 msgr2=0x557bf3a49cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a49cb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a40013270 tx=0x7f0a40005bd0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 -- 172.21.15.37:0/4267585882 shutdown_connections 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a4a800 0x557bf3a49cb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3ff40 0x557bf3a42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 --2- 172.21.15.37:0/4267585882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a3f620 0x557bf3a56290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 -- 172.21.15.37:0/4267585882 >> 172.21.15.37:0/4267585882 conn(0x557bf3a3bb30 msgr2=0x557bf3a42260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 -- 172.21.15.37:0/4267585882 shutdown_connections 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.989+0000 7f0a525da740 1 -- 172.21.15.37:0/4267585882 wait complete. 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 -- start start 2024-04-23T18:24:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 Processor -- start 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a52950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 0x557bf3a4dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a4a800 0x557bf3a4e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481ae460 con 0x557bf3a3ff40 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481af8d0 con 0x557bf3a4a800 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481aeb30 con 0x557bf3a3f620 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a525da740 10 client.0 _ll_register_callbacks cb 0x7f0a44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a4cd80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 0x557bf3a4dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a4d581640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a52950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a4cd80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 0x557bf3a4dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51302/0 (socket says 172.21.15.37:51302) 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a4cd80640 1 -- 172.21.15.37:0/2258994981 learned_addr learned my addr 172.21.15.37:0/2258994981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.990+0000 7f0a4dd82640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a4a800 0x557bf3a4e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4d581640 1 -- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a4a800 msgr2=0x557bf3a4e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4d581640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a4a800 0x557bf3a4e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4d581640 1 -- 172.21.15.37:0/2258994981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 msgr2=0x557bf3a4dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4d581640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 0x557bf3a4dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4d581640 1 -- 172.21.15.37:0/2258994981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf3a4e900 con 0x557bf3a3f620 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 0x557bf3a4dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a4d581640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a52950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a4402a9a0 tx=0x7f0a44004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1280ca69c0cf1546 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a30ff9640 1 -- 172.21.15.37:0/2258994981 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4404c050 con 0x557bf3a3f620 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a30ff9640 1 -- 172.21.15.37:0/2258994981 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0a44031870 con 0x557bf3a3f620 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40010430 con 0x557bf3a3f620 2024-04-23T18:24:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a30ff9640 1 -- 172.21.15.37:0/2258994981 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0a44040050 con 0x557bf3a3f620 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a525da740 10 client.25600 fetch_fsmap learned FSMap version 30 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557bf3ac0770 con 0x557bf3a3f620 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a30ff9640 1 -- 172.21.15.37:0/2258994981 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40010430 con 0x557bf3a3f620 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.991+0000 7f0a30ff9640 1 -- 172.21.15.37:0/2258994981 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0a44003a70 con 0x557bf3a3f620 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a525da740 10 client.25600 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a525da740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a525da740 20 client.25600 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a525da740 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3ac0e80 0x557bf3ac32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a525da740 4 client.25600 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bf39ac200 con 0x557bf3ac0e80 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3ac0e80 0x557bf3ac32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a30ff9640 10 client.25600 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:42.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:42.992+0000 7f0a4cd80640 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3ac0e80 0x557bf3ac32a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb14d4253c985706 server_cookie=444e3377c2f253b4 in_seq=0 out_seq=0 2024-04-23T18:24:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a30ff9640 1 -- 172.21.15.37:0/2258994981 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bf39ac200 con 0x557bf3ac0e80 2024-04-23T18:24:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a30ff9640 10 client.25600 handle_command_reply: tid=0 2024-04-23T18:24:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 client.25600 shutdown 2024-04-23T18:24:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 msgr2=0x557bf3a52950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a52950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a4402a9a0 tx=0x7f0a44004470 comp rx=0 tx=0).stop 2024-04-23T18:24:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 shutdown_connections 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557bf3ac0e80 0x557bf3ac32a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bf3a4a800 0x557bf3a4e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf3a3ff40 0x557bf3a4dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 --2- 172.21.15.37:0/2258994981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557bf3a3f620 0x557bf3a52950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a30ff9640 0 client.25600 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.048+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 >> 172.21.15.37:0/2258994981 conn(0x557bf3a3bb30 msgr2=0x557bf3a3acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 shutdown_connections 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a525da740 1 -- 172.21.15.37:0/2258994981 wait complete. 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a525da740 20 client.25600 trim_cache size 0 max 16384 2024-04-23T18:24:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:24:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:24:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:24:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:24:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0a340a3e40 msgr2=0x7f0a340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0a340a3e40 0x7f0a340a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a400049d0 tx=0x7f0a40004ab0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 msgr2=0x7f0a481a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 0x7f0a481a0b40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800e9f0 tx=0x7f0a3800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 shutdown_connections 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0a340a3e40 0x7f0a340a6300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a4810c5a0 0x7f0a4819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a48103af0 0x7f0a481a0b40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 --2- 172.21.15.37:0/2741922299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a481031d0 0x7f0a481a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 >> 172.21.15.37:0/2741922299 conn(0x7f0a48077fa0 msgr2=0x7f0a48108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 shutdown_connections 2024-04-23T18:24:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:43.049+0000 7f0a4f7e5640 1 -- 172.21.15.37:0/2741922299 wait complete. 2024-04-23T18:24:43.355 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:24:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: ignoring --setuser ceph since I am not root 2024-04-23T18:24:43.355 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:24:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: ignoring --setgroup ceph since I am not root 2024-04-23T18:24:43.355 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:24:43 smithi173 ceph-mds[63204]: -- 172.21.15.173:0/1819461063 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555637c31a40 con 0x555637c04800 2024-04-23T18:24:43.355 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:24:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: starting mds.c at 2024-04-23T18:24:43.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 274095 3.85 MB/sec execute 48 sec latency 44.311 ms 2024-04-23T18:24:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:43 smithi172 ceph-mon[32456]: pgmap v131: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-04-23T18:24:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:43 smithi172 ceph-mon[32456]: daemon mds.c restarted 2024-04-23T18:24:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:43 smithi173 ceph-mon[32906]: pgmap v131: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-04-23T18:24:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:43 smithi173 ceph-mon[32906]: daemon mds.c restarted 2024-04-23T18:24:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:43 smithi037 ceph-mon[29404]: pgmap v131: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-04-23T18:24:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:43 smithi037 ceph-mon[29404]: daemon mds.c restarted 2024-04-23T18:24:44.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 276767 4.12 MB/sec execute 49 sec latency 44.079 ms 2024-04-23T18:24:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:44 smithi172 ceph-mon[32456]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:24:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:44 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6836/3140945410,v1:172.21.15.173:6837/3140945410] up:boot 2024-04-23T18:24:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:44 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-04-23T18:24:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:44 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:24:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:44 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:45.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:44 smithi173 ceph-mon[32906]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:24:45.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:44 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6836/3140945410,v1:172.21.15.173:6837/3140945410] up:boot 2024-04-23T18:24:45.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:44 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-04-23T18:24:45.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:44 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:24:45.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:44 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:44 smithi037 ceph-mon[29404]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:24:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:44 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6836/3140945410,v1:172.21.15.173:6837/3140945410] up:boot 2024-04-23T18:24:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:44 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-04-23T18:24:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:44 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:24:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:44 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:24:45.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 278078 4.16 MB/sec execute 50 sec latency 16.881 ms 2024-04-23T18:24:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:45 smithi172 ceph-mon[32456]: pgmap v133: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-04-23T18:24:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:45 smithi173 ceph-mon[32906]: pgmap v133: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-04-23T18:24:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:45 smithi037 ceph-mon[29404]: pgmap v133: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-04-23T18:24:46.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 280510 4.35 MB/sec execute 51 sec latency 16.863 ms 2024-04-23T18:24:47.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 282198 4.38 MB/sec execute 52 sec latency 11.658 ms 2024-04-23T18:24:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:47 smithi172 ceph-mon[32456]: pgmap v134: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:47 smithi173 ceph-mon[32906]: pgmap v134: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:47 smithi037 ceph-mon[29404]: pgmap v134: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:48.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 284186 4.55 MB/sec execute 53 sec latency 33.348 ms 2024-04-23T18:24:49.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 286234 4.60 MB/sec execute 54 sec latency 17.493 ms 2024-04-23T18:24:50.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:49 smithi172 ceph-mon[32456]: pgmap v135: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:49 smithi173 ceph-mon[32906]: pgmap v135: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:49 smithi037 ceph-mon[29404]: pgmap v135: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:50.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 287890 4.74 MB/sec execute 55 sec latency 10.986 ms 2024-04-23T18:24:51.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 290323 4.88 MB/sec execute 56 sec latency 19.669 ms 2024-04-23T18:24:52.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:51 smithi172 ceph-mon[32456]: pgmap v136: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:51 smithi173 ceph-mon[32906]: pgmap v136: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:51 smithi037 ceph-mon[29404]: pgmap v136: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-23T18:24:52.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 291789 4.92 MB/sec execute 57 sec latency 15.291 ms 2024-04-23T18:24:53.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 294537 5.13 MB/sec execute 58 sec latency 21.955 ms 2024-04-23T18:24:54.156 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:53 smithi173 ceph-mon[32906]: pgmap v137: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-04-23T18:24:54.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:53 smithi172 ceph-mon[32456]: pgmap v137: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-04-23T18:24:54.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:53 smithi037 ceph-mon[29404]: pgmap v137: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-04-23T18:24:54.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 295984 5.15 MB/sec execute 59 sec latency 17.535 ms 2024-04-23T18:24:55.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298344 5.30 MB/sec execute 60 sec latency 13.401 ms 2024-04-23T18:24:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2679202692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd94103a20 msgr2=0x7fbd9410bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2679202692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd94103a20 0x7fbd9410bf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009860 tx=0x7fbd8002a3d0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2679202692 shutdown_connections 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2679202692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd9410c520 0x7fbd9410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2679202692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd94103a20 0x7fbd9410bf00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2679202692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2679202692 >> 172.21.15.37:0/2679202692 conn(0x7fbd940fc920 msgr2=0x7fbd940fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2679202692 shutdown_connections 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.142+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2679202692 wait complete. 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 Processor -- start 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 -- start start 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd94103a20 0x7fbd941a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd9410c520 0x7fbd9419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941121e0 con 0x7fbd94103100 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94112060 con 0x7fbd94103a20 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9aecf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94112360 con 0x7fbd9410c520 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd98c6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd98c6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47858/0 (socket says 172.21.15.37:47858) 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd98c6b640 1 -- 172.21.15.37:0/2153500528 learned_addr learned my addr 172.21.15.37:0/2153500528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd94103a20 0x7fbd941a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.143+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd9410c520 0x7fbd9419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd98c6b640 1 -- 172.21.15.37:0/2153500528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd9410c520 msgr2=0x7fbd9419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd98c6b640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd9410c520 0x7fbd9419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd98c6b640 1 -- 172.21.15.37:0/2153500528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd94103a20 msgr2=0x7fbd941a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd98c6b640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd94103a20 0x7fbd941a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd98c6b640 1 -- 172.21.15.37:0/2153500528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9419b0e0 con 0x7fbd94103100 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd98c6b640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941a06d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800e9c0 tx=0x7fbd8800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd91ffb640 1 -- 172.21.15.37:0/2153500528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800ccc0 con 0x7fbd94103100 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9419b3d0 con 0x7fbd94103100 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd91ffb640 1 -- 172.21.15.37:0/2153500528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd880054d0 con 0x7fbd94103100 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd91ffb640 1 -- 172.21.15.37:0/2153500528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88010430 con 0x7fbd94103100 2024-04-23T18:24:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.144+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941a76c0 con 0x7fbd94103100 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.147+0000 7fbd9dbe7740 1 Processor -- start 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.147+0000 7fbd91ffb640 1 -- 172.21.15.37:0/2153500528 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd88004f20 con 0x7fbd94103100 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.147+0000 7fbd9dbe7740 1 -- start start 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af147620 0x5588af147a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af152800 0x5588af154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5588af0ac970 con 0x5588af152800 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588af0494a0 con 0x5588af147620 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5588af0a8930 con 0x5588af147f40 2024-04-23T18:24:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd93fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af152800 0x5588af154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af147620 0x5588af147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd98c6b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd93fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af152800 0x5588af154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47870/0 (socket says 172.21.15.37:47870) 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd93fff640 1 -- 172.21.15.37:0/2781093826 learned_addr learned my addr 172.21.15.37:0/2781093826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 -- 172.21.15.37:0/2781093826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 msgr2=0x5588af1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 -- 172.21.15.37:0/2781093826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af152800 msgr2=0x5588af154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af152800 0x5588af154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 -- 172.21.15.37:0/2781093826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588aedbf210 con 0x5588af147620 2024-04-23T18:24:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd91ffb640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbd840a3dc0 0x7fbd840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af147620 0x5588af147a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009f10 tx=0x7fbd8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370c2532a08c53dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.148+0000 7fbd98c6b640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbd840a3dc0 0x7fbd840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.149+0000 7fbd91ffb640 1 -- 172.21.15.37:0/2153500528 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbd880d0ce0 con 0x7fbd94103100 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.149+0000 7fbd7effd640 1 -- 172.21.15.37:0/2781093826 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c004d20 con 0x5588af147620 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.149+0000 7fbd7effd640 1 -- 172.21.15.37:0/2781093826 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd8c004050 con 0x5588af147620 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.149+0000 7fbd7effd640 1 -- 172.21.15.37:0/2781093826 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c005220 con 0x5588af147620 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2781093826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af147620 msgr2=0x5588af147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd98c6b640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbd840a3dc0 0x7fbd840a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd880ce4d0 tx=0x7fbd880bd040 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af147620 0x5588af147a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009f10 tx=0x7fbd8c011810 comp rx=0 tx=0).stop 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2781093826 shutdown_connections 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af152800 0x5588af154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2781093826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af147620 0x5588af147a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2781093826 >> 172.21.15.37:0/2781093826 conn(0x5588af143b30 msgr2=0x5588af143f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2781093826 shutdown_connections 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2781093826 wait complete. 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 -- start start 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.150+0000 7fbd9dbe7740 1 Processor -- start 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af152180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:40550/0 (socket says 172.21.15.37:40550) 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 -- 172.21.15.37:0/2491318168 learned_addr learned my addr 172.21.15.37:0/2491318168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5588af155430 con 0x5588af147f40 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588af0470b0 con 0x5588af152800 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5588af1555b0 con 0x5588af147620 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9dbe7740 10 client.0 _ll_register_callbacks cb 0x5588af034d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 -- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 msgr2=0x5588af14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 -- 172.21.15.37:0/2491318168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 msgr2=0x5588af14f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588aee8b620 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af152180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c01b040 tx=0x7fbd8c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25496f5a5b35e78f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.151+0000 7fbd73fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd73fff640 1 -- 172.21.15.37:0/2491318168 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c003c30 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd73fff640 1 -- 172.21.15.37:0/2491318168 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbd8c003dd0 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd8c004050 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd73fff640 1 -- 172.21.15.37:0/2491318168 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd8c005040 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 10 client.25618 fetch_fsmap learned FSMap version 32 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5588af161360 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd73fff640 1 -- 172.21.15.37:0/2491318168 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd8c004050 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd73fff640 1 -- 172.21.15.37:0/2491318168 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbd8c0285e0 con 0x5588af147620 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 10 client.25618 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 20 client.25618 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1627c0 0x5588af164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 4 client.25618 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:24:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.152+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588af0ba390 con 0x5588af1627c0 2024-04-23T18:24:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.153+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1627c0 0x5588af164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.153+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1627c0 0x5588af164bb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86fbff537881dea4 server_cookie=c0566e9d5399a629 in_seq=0 out_seq=0 2024-04-23T18:24:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.153+0000 7fbd73fff640 10 client.25618 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd73fff640 1 -- 172.21.15.37:0/2491318168 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588af0ba390 con 0x5588af1627c0 2024-04-23T18:24:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd73fff640 10 client.25618 handle_command_reply: tid=0 2024-04-23T18:24:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 client.25618 shutdown 2024-04-23T18:24:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 msgr2=0x5588af152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af152180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c01b040 tx=0x7fbd8c004790 comp rx=0 tx=0).stop 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 shutdown_connections 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1627c0 0x5588af164bb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2491318168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af152180 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 >> 172.21.15.37:0/2491318168 conn(0x5588af143b30 msgr2=0x5588af149860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd73fff640 0 client.25618 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 shutdown_connections 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2491318168 wait complete. 2024-04-23T18:24:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.154+0000 7fbd9dbe7740 20 client.25618 trim_cache size 0 max 16384 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 Processor -- start 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 -- start start 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b2740 con 0x5588af147f40 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941affb0 con 0x5588af152800 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941ade00 con 0x5588af147620 2024-04-23T18:24:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd93fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9946c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd93fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47896/0 (socket says 172.21.15.37:47896) 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd93fff640 1 -- 172.21.15.37:0/2189717940 learned_addr learned my addr 172.21.15.37:0/2189717940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.171+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd93fff640 1 -- 172.21.15.37:0/2189717940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 msgr2=0x5588af15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd93fff640 1 -- 172.21.15.37:0/2189717940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 msgr2=0x5588af151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd93fff640 1 -- 172.21.15.37:0/2189717940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588af147d70 con 0x5588af147f40 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f410 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbd80032ff0 tx=0x7fbd8002ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e7ec544afd0e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd7e7fc640 1 -- 172.21.15.37:0/2189717940 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd800306a0 con 0x5588af147f40 2024-04-23T18:24:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd7e7fc640 1 -- 172.21.15.37:0/2189717940 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80030980 con 0x5588af147f40 2024-04-23T18:24:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.172+0000 7fbd7e7fc640 1 -- 172.21.15.37:0/2189717940 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80046630 con 0x5588af147f40 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2189717940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 msgr2=0x5588af14f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f410 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbd80032ff0 tx=0x7fbd8002ab70 comp rx=0 tx=0).stop 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2189717940 shutdown_connections 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147f40 0x5588af14f410 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2189717940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147620 0x5588af15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2189717940 >> 172.21.15.37:0/2189717940 conn(0x5588af143b30 msgr2=0x5588af14ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2189717940 shutdown_connections 2024-04-23T18:24:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2189717940 wait complete. 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 -- start start 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.173+0000 7fbd9dbe7740 1 Processor -- start 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 0x5588af15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941ae470 con 0x5588af147620 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941af8e0 con 0x5588af152800 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941aeb40 con 0x5588af147f40 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9dbe7740 10 client.0 _ll_register_callbacks cb 0x7fbd8c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 0x5588af15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 0x5588af15acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47908/0 (socket says 172.21.15.37:47908) 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 -- 172.21.15.37:0/2750297739 learned_addr learned my addr 172.21.15.37:0/2750297739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:24:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 -- 172.21.15.37:0/2750297739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 msgr2=0x5588af14cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af14cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 -- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 msgr2=0x5588af14d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 -- 172.21.15.37:0/2750297739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588af14b9a0 con 0x5588af147620 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd9946c640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 0x5588af15acd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c004fd0 tx=0x7fbd8c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c528d104e26f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd73fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.174+0000 7fbd73fff640 1 -- 172.21.15.37:0/2750297739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c046060 con 0x5588af147620 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd73fff640 1 -- 172.21.15.37:0/2750297739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbd8c003a30 con 0x5588af147620 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd80030980 con 0x5588af147620 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd73fff640 1 -- 172.21.15.37:0/2750297739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd8c003c50 con 0x5588af147620 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 10 client.16083 fetch_fsmap learned FSMap version 32 2024-04-23T18:24:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5588af1c8130 con 0x5588af147620 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd73fff640 1 -- 172.21.15.37:0/2750297739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80030980 con 0x5588af147620 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd73fff640 1 -- 172.21.15.37:0/2750297739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbd8c005820 con 0x5588af147620 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 10 client.16083 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 20 client.16083 populate_metadata read hostname 'smithi037' 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1c8770 0x5588af1cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 4 client.16083 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588af0b4200 con 0x5588af1c8770 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.175+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1c8770 0x5588af1cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.176+0000 7fbd93fff640 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1c8770 0x5588af1cab90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c51f48f5feb237 server_cookie=82bb7194b5502411 in_seq=0 out_seq=0 2024-04-23T18:24:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.176+0000 7fbd73fff640 10 client.16083 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd73fff640 1 -- 172.21.15.37:0/2750297739 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588af0b4200 con 0x5588af1c8770 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd73fff640 10 client.16083 handle_command_reply: tid=0 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 client.16083 shutdown 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 msgr2=0x5588af15acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 0x5588af15acd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c004fd0 tx=0x7fbd8c004790 comp rx=0 tx=0).stop 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 shutdown_connections 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5588af1c8770 0x5588af1cab90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588af152800 0x5588af14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588af147f40 0x5588af14cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 --2- 172.21.15.37:0/2750297739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588af147620 0x5588af15acd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd73fff640 0 client.16083 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.242+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 >> 172.21.15.37:0/2750297739 conn(0x5588af143b30 msgr2=0x5588af148fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 shutdown_connections 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9dbe7740 1 -- 172.21.15.37:0/2750297739 wait complete. 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9dbe7740 20 client.16083 trim_cache size 0 max 16384 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:24:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:24:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbd840a3dc0 msgr2=0x7fbd840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbd840a3dc0 0x7fbd840a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd880ce4d0 tx=0x7fbd880bd040 comp rx=0 tx=0).stop 2024-04-23T18:24:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 msgr2=0x7fbd941a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941a06d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800e9c0 tx=0x7fbd8800edc0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 shutdown_connections 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbd840a3dc0 0x7fbd840a6280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd9410c520 0x7fbd9419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd94103a20 0x7fbd941a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 --2- 172.21.15.37:0/2153500528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd94103100 0x7fbd941a06d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 >> 172.21.15.37:0/2153500528 conn(0x7fbd940fc920 msgr2=0x7fbd94100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 shutdown_connections 2024-04-23T18:24:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:24:56.243+0000 7fbd9aecf640 1 -- 172.21.15.37:0/2153500528 wait complete. 2024-04-23T18:24:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:55 smithi172 ceph-mon[32456]: pgmap v138: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 272 op/s 2024-04-23T18:24:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:55 smithi173 ceph-mon[32906]: pgmap v138: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 272 op/s 2024-04-23T18:24:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:55 smithi037 ceph-mon[29404]: pgmap v138: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 272 op/s 2024-04-23T18:24:56.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 300376 5.33 MB/sec execute 61 sec latency 39.498 ms 2024-04-23T18:24:57.817 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302111 5.44 MB/sec execute 62 sec latency 10.497 ms 2024-04-23T18:24:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:57 smithi172 ceph-mon[32456]: pgmap v139: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 384 op/s 2024-04-23T18:24:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:57 smithi173 ceph-mon[32906]: pgmap v139: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 384 op/s 2024-04-23T18:24:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:57 smithi037 ceph-mon[29404]: pgmap v139: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 384 op/s 2024-04-23T18:24:58.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 304521 5.55 MB/sec execute 63 sec latency 16.310 ms 2024-04-23T18:24:59.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 305968 5.58 MB/sec execute 64 sec latency 13.468 ms 2024-04-23T18:25:00.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:24:59 smithi172 ceph-mon[32456]: pgmap v140: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-04-23T18:25:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:24:59 smithi173 ceph-mon[32906]: pgmap v140: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-04-23T18:25:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:24:59 smithi037 ceph-mon[29404]: pgmap v140: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-04-23T18:25:00.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 308271 5.68 MB/sec execute 65 sec latency 52.415 ms 2024-04-23T18:25:01.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 309173 5.69 MB/sec execute 66 sec latency 21.384 ms 2024-04-23T18:25:02.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:01 smithi172 ceph-mon[32456]: pgmap v141: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-04-23T18:25:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:01 smithi173 ceph-mon[32906]: pgmap v141: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-04-23T18:25:02.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:01 smithi037 ceph-mon[29404]: pgmap v141: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-04-23T18:25:02.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 311409 5.77 MB/sec execute 67 sec latency 10.683 ms 2024-04-23T18:25:03.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 312945 5.81 MB/sec execute 68 sec latency 15.518 ms 2024-04-23T18:25:04.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:03 smithi037 ceph-mon[29404]: pgmap v142: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-04-23T18:25:04.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:03 smithi172 ceph-mon[32456]: pgmap v142: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-04-23T18:25:04.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:03 smithi173 ceph-mon[32906]: pgmap v142: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-04-23T18:25:04.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 315525 5.93 MB/sec execute 69 sec latency 16.912 ms 2024-04-23T18:25:05.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 316685 5.92 MB/sec execute 70 sec latency 52.458 ms 2024-04-23T18:25:06.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:05 smithi172 ceph-mon[32456]: pgmap v143: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-04-23T18:25:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:05 smithi173 ceph-mon[32906]: pgmap v143: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-04-23T18:25:06.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:05 smithi037 ceph-mon[29404]: pgmap v143: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-04-23T18:25:06.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.89 MB/sec execute 71 sec latency 864.528 ms 2024-04-23T18:25:07.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.81 MB/sec execute 72 sec latency 1864.622 ms 2024-04-23T18:25:08.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:07 smithi172 ceph-mon[32456]: pgmap v144: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-23T18:25:08.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:07 smithi173 ceph-mon[32906]: pgmap v144: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-23T18:25:08.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:07 smithi037 ceph-mon[29404]: pgmap v144: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-23T18:25:08.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.73 MB/sec execute 73 sec latency 2864.729 ms 2024-04-23T18:25:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 -- 172.21.15.37:0/236712760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c1010f0 msgr2=0x7fed5c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 --2- 172.21.15.37:0/236712760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c1010f0 0x7fed5c1014d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009900 tx=0x7fed4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 -- 172.21.15.37:0/236712760 shutdown_connections 2024-04-23T18:25:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 --2- 172.21.15.37:0/236712760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 0x7fed5c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 --2- 172.21.15.37:0/236712760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c101aa0 0x7fed5c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 --2- 172.21.15.37:0/236712760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c1010f0 0x7fed5c1014d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 -- 172.21.15.37:0/236712760 >> 172.21.15.37:0/236712760 conn(0x7fed5c077fd0 msgr2=0x7fed5c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 -- 172.21.15.37:0/236712760 shutdown_connections 2024-04-23T18:25:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.343+0000 7fed61e72640 1 -- 172.21.15.37:0/236712760 wait complete. 2024-04-23T18:25:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 Processor -- start 2024-04-23T18:25:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 -- start start 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c1010f0 0x7fed5c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 0x7fed5c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 0x7fed5c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1121f0 con 0x7fed5c19cfb0 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112070 con 0x7fed5c106170 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed61e72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c112370 con 0x7fed5c1010f0 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 0x7fed5c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 0x7fed5c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37628/0 (socket says 172.21.15.37:37628) 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed5bfff640 1 -- 172.21.15.37:0/3670051838 learned_addr learned my addr 172.21.15.37:0/3670051838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed5affd640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 0x7fed5c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.344+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c1010f0 0x7fed5c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5bfff640 1 -- 172.21.15.37:0/3670051838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c1010f0 msgr2=0x7fed5c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5bfff640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c1010f0 0x7fed5c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5bfff640 1 -- 172.21.15.37:0/3670051838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 msgr2=0x7fed5c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5bfff640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 0x7fed5c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5bfff640 1 -- 172.21.15.37:0/3670051838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c19daa0 con 0x7fed5c19cfb0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5affd640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 0x7fed5c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c1010f0 0x7fed5c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed5bfff640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 0x7fed5c19d460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fed5400cad0 tx=0x7fed5400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed58ff9640 1 -- 172.21.15.37:0/3670051838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5400ae20 con 0x7fed5c19cfb0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed58ff9640 1 -- 172.21.15.37:0/3670051838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed54005020 con 0x7fed5c19cfb0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c19dd90 con 0x7fed5c19cfb0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed58ff9640 1 -- 172.21.15.37:0/3670051838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5400f6b0 con 0x7fed5c19cfb0 2024-04-23T18:25:09.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.345+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1a5580 con 0x7fed5c19cfb0 2024-04-23T18:25:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed58ff9640 1 -- 172.21.15.37:0/3670051838 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed540051d0 con 0x7fed5c19cfb0 2024-04-23T18:25:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 Processor -- start 2024-04-23T18:25:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 -- start start 2024-04-23T18:25:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a16900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a28000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56306197b970 con 0x563061a16e40 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630619184a0 con 0x563061a16520 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed64b89740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563061977930 con 0x563061a25c10 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed58ff9640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fed440a3e40 0x7fed440a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37640/0 (socket says 172.21.15.37:37640) 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed5b7fe640 1 -- 172.21.15.37:0/3818945483 learned_addr learned my addr 172.21.15.37:0/3818945483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed5bfff640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fed440a3e40 0x7fed440a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.348+0000 7fed5affd640 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a28000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5affd640 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a16900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed58ff9640 1 -- 172.21.15.37:0/3670051838 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fed540d08e0 con 0x7fed5c19cfb0 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5b7fe640 1 -- 172.21.15.37:0/3818945483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 msgr2=0x563061a28000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a28000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5b7fe640 1 -- 172.21.15.37:0/3818945483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 msgr2=0x563061a16900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a16900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5b7fe640 1 -- 172.21.15.37:0/3818945483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306168e210 con 0x563061a16e40 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5bfff640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fed440a3e40 0x7fed440a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fed5400caa0 tx=0x7fed54019f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a256d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fed4c02a930 tx=0x7fed4c032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96dfb78e17df15 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed41ffb640 1 -- 172.21.15.37:0/3818945483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c03f070 con 0x563061a16e40 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed41ffb640 1 -- 172.21.15.37:0/3818945483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4c0316a0 con 0x563061a16e40 2024-04-23T18:25:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.349+0000 7fed41ffb640 1 -- 172.21.15.37:0/3818945483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c035070 con 0x563061a16e40 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 -- 172.21.15.37:0/3818945483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 msgr2=0x563061a256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a256d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fed4c02a930 tx=0x7fed4c032e70 comp rx=0 tx=0).stop 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 -- 172.21.15.37:0/3818945483 shutdown_connections 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a28000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a256d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 --2- 172.21.15.37:0/3818945483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a16900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 -- 172.21.15.37:0/3818945483 >> 172.21.15.37:0/3818945483 conn(0x563061a12b30 msgr2=0x563061a12f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 -- 172.21.15.37:0/3818945483 shutdown_connections 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 -- 172.21.15.37:0/3818945483 wait complete. 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.350+0000 7fed64b89740 1 -- start start 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 Processor -- start 2024-04-23T18:25:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563061a287e0 con 0x563061a16e40 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630619160b0 con 0x563061a16520 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563061a28960 con 0x563061a25c10 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed64b89740 10 client.0 _ll_register_callbacks cb 0x563061903d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37642/0 (socket says 172.21.15.37:37642) 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1275282854 learned_addr learned my addr 172.21.15.37:0/1275282854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5affd640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5affd640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1275282854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 msgr2=0x563061a22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a22cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 msgr2=0x563061a254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1275282854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306175a620 con 0x563061a16e40 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.351+0000 7fed5affd640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a22790 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006010 tx=0x7fed4c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5058bad4a174197 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed3affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed3affd640 1 -- 172.21.15.37:0/1275282854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c03f070 con 0x563061a16e40 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed3affd640 1 -- 172.21.15.37:0/1275282854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fed4c004440 con 0x563061a16e40 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed4c0316a0 con 0x563061a16e40 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed3affd640 1 -- 172.21.15.37:0/1275282854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed4c054050 con 0x563061a16e40 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 10 client.16095 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563061a346a0 con 0x563061a16e40 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed3affd640 1 -- 172.21.15.37:0/1275282854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4c0316a0 con 0x563061a16e40 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed3affd640 1 -- 172.21.15.37:0/1275282854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fed4c004a00 con 0x563061a16e40 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 10 client.16095 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 20 client.16095 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a359f0 0x563061a37de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 4 client.16095 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.352+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563061989390 con 0x563061a359f0 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.353+0000 7fed5affd640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a359f0 0x563061a37de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.353+0000 7fed3affd640 10 client.16095 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.353+0000 7fed5affd640 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a359f0 0x563061a37de0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580fdaee919eb55e server_cookie=57a5978e126cd98c in_seq=0 out_seq=0 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed3affd640 1 -- 172.21.15.37:0/1275282854 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563061989390 con 0x563061a359f0 2024-04-23T18:25:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed3affd640 10 client.16095 handle_command_reply: tid=0 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 client.16095 shutdown 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 msgr2=0x563061a22790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a22790 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006010 tx=0x7fed4c005230 comp rx=0 tx=0).stop 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 shutdown_connections 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a359f0 0x563061a37de0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a22790 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 --2- 172.21.15.37:0/1275282854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed3affd640 0 client.16095 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 >> 172.21.15.37:0/1275282854 conn(0x563061a12b30 msgr2=0x563061a189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 shutdown_connections 2024-04-23T18:25:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.354+0000 7fed64b89740 1 -- 172.21.15.37:0/1275282854 wait complete. 2024-04-23T18:25:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.355+0000 7fed64b89740 20 client.16095 trim_cache size 0 max 16384 2024-04-23T18:25:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 Processor -- start 2024-04-23T18:25:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 -- start start 2024-04-23T18:25:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a16520 0x563061a316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a1dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a25c10 0x563061a24fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1ae290 con 0x563061a16e40 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1abb00 con 0x563061a25c10 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed64b89740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1a9950 con 0x563061a16520 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a1dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a1dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37654/0 (socket says 172.21.15.37:37654) 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.370+0000 7fed5b7fe640 1 -- 172.21.15.37:0/2738601876 learned_addr learned my addr 172.21.15.37:0/2738601876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5affd640 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a16520 0x563061a316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5affd640 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a25c10 0x563061a24fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5b7fe640 1 -- 172.21.15.37:0/2738601876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a16520 msgr2=0x563061a316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a16520 0x563061a316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5b7fe640 1 -- 172.21.15.37:0/2738601876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a25c10 msgr2=0x563061a24fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a25c10 0x563061a24fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5b7fe640 1 -- 172.21.15.37:0/2738601876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563061a12960 con 0x563061a16e40 2024-04-23T18:25:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a1dd80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fed4c033000 tx=0x7fed4c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d096a241609696e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed417fa640 1 -- 172.21.15.37:0/2738601876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c05d070 con 0x563061a16e40 2024-04-23T18:25:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed417fa640 1 -- 172.21.15.37:0/2738601876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4c04b070 con 0x563061a16e40 2024-04-23T18:25:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.371+0000 7fed417fa640 1 -- 172.21.15.37:0/2738601876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c03f040 con 0x563061a16e40 2024-04-23T18:25:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 -- 172.21.15.37:0/2738601876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 msgr2=0x563061a1dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a1dd80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fed4c033000 tx=0x7fed4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 -- 172.21.15.37:0/2738601876 shutdown_connections 2024-04-23T18:25:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a25c10 0x563061a24fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a1dd80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 --2- 172.21.15.37:0/2738601876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a16520 0x563061a316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 -- 172.21.15.37:0/2738601876 >> 172.21.15.37:0/2738601876 conn(0x563061a12b30 msgr2=0x563061a1d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 -- 172.21.15.37:0/2738601876 shutdown_connections 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 -- 172.21.15.37:0/2738601876 wait complete. 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 -- start start 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.372+0000 7fed64b89740 1 Processor -- start 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a29470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a29bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1a9fc0 con 0x563061a16e40 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1ab430 con 0x563061a16520 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1aa690 con 0x563061a25c10 2024-04-23T18:25:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed64b89740 10 client.0 _ll_register_callbacks cb 0x7fed4c005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a29470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a29470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37656/0 (socket says 172.21.15.37:37656) 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1314445681 learned_addr learned my addr 172.21.15.37:0/1314445681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5affd640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5affd640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a29bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1314445681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 msgr2=0x563061a29bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a29bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 msgr2=0x563061a2e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a2e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 -- 172.21.15.37:0/1314445681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563061a12550 con 0x563061a16e40 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5affd640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a2e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed5b7fe640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a29470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005420 tx=0x7fed4c032af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b173c370087b9196 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed3affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.373+0000 7fed3affd640 1 -- 172.21.15.37:0/1314445681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c03f070 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed3affd640 1 -- 172.21.15.37:0/1314445681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fed4c031a60 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed4c04b070 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed3affd640 1 -- 172.21.15.37:0/1314445681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed4c05d070 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 10 client.16101 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563061a17380 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed3affd640 1 -- 172.21.15.37:0/1314445681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4c04b070 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed3affd640 1 -- 172.21.15.37:0/1314445681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fed4c060020 con 0x563061a16e40 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 10 client.16101 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 20 client.16101 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a9bf40 0x563061a9e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 4 client.16101 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:25:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563061a23060 con 0x563061a9bf40 2024-04-23T18:25:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.374+0000 7fed5affd640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a9bf40 0x563061a9e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.375+0000 7fed5affd640 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a9bf40 0x563061a9e360 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf080ace5645fc server_cookie=275c2fd50d207e3e in_seq=0 out_seq=0 2024-04-23T18:25:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:09.375+0000 7fed3affd640 10 client.16101 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:09.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.66 MB/sec execute 74 sec latency 3864.799 ms 2024-04-23T18:25:10.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:09 smithi172 ceph-mon[32456]: pgmap v145: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-04-23T18:25:10.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:09 smithi173 ceph-mon[32906]: pgmap v145: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-04-23T18:25:10.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:09 smithi037 ceph-mon[29404]: pgmap v145: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-04-23T18:25:10.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.58 MB/sec execute 75 sec latency 4864.855 ms 2024-04-23T18:25:11.818 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.51 MB/sec execute 76 sec latency 5864.982 ms 2024-04-23T18:25:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:11 smithi172 ceph-mon[32456]: pgmap v146: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-04-23T18:25:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:11 smithi173 ceph-mon[32906]: pgmap v146: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-04-23T18:25:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:11 smithi037 ceph-mon[29404]: pgmap v146: 97 pgs: 97 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-04-23T18:25:12.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.44 MB/sec execute 77 sec latency 6865.107 ms 2024-04-23T18:25:13.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.37 MB/sec execute 78 sec latency 7865.172 ms 2024-04-23T18:25:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:13 smithi172 ceph-mon[32456]: pgmap v147: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2024-04-23T18:25:14.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:13 smithi173 ceph-mon[32906]: pgmap v147: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2024-04-23T18:25:14.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:13 smithi037 ceph-mon[29404]: pgmap v147: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 302 op/s 2024-04-23T18:25:14.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.30 MB/sec execute 79 sec latency 8865.290 ms 2024-04-23T18:25:15.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.23 MB/sec execute 80 sec latency 9865.409 ms 2024-04-23T18:25:16.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:15 smithi172 ceph-mon[32456]: pgmap v148: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-04-23T18:25:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:15 smithi173 ceph-mon[32906]: pgmap v148: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-04-23T18:25:16.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:15 smithi037 ceph-mon[29404]: pgmap v148: 97 pgs: 97 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-04-23T18:25:16.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317105 5.17 MB/sec execute 81 sec latency 10865.525 ms 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.609+0000 7fed3affd640 1 -- 172.21.15.37:0/1314445681 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563061a23060 con 0x563061a9bf40 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.609+0000 7fed3affd640 10 client.16101 handle_command_reply: tid=0 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.609+0000 7fed64b89740 1 client.16101 shutdown 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 msgr2=0x563061a29470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a29470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005420 tx=0x7fed4c032af0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 shutdown_connections 2024-04-23T18:25:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563061a9bf40 0x563061a9e360 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563061a25c10 0x563061a29bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563061a16e40 0x563061a29470 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 --2- 172.21.15.37:0/1314445681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563061a16520 0x563061a2e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed3affd640 0 client.16101 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 >> 172.21.15.37:0/1314445681 conn(0x563061a12b30 msgr2=0x563061a18110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 shutdown_connections 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 1 -- 172.21.15.37:0/1314445681 wait complete. 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.610+0000 7fed64b89740 20 client.16101 trim_cache size 0 max 16384 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:25:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:25:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fed440a3e40 msgr2=0x7fed440a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fed440a3e40 0x7fed440a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fed5400caa0 tx=0x7fed54019f90 comp rx=0 tx=0).stop 2024-04-23T18:25:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 msgr2=0x7fed5c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 0x7fed5c19d460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fed5400cad0 tx=0x7fed5400cf90 comp rx=0 tx=0).stop 2024-04-23T18:25:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 shutdown_connections 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fed440a3e40 0x7fed440a6300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed5c19cfb0 0x7fed5c19d460 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5c106170 0x7fed5c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 --2- 172.21.15.37:0/3670051838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed5c1010f0 0x7fed5c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 >> 172.21.15.37:0/3670051838 conn(0x7fed5c077fd0 msgr2=0x7fed5c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 shutdown_connections 2024-04-23T18:25:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:17.611+0000 7fed61e72640 1 -- 172.21.15.37:0/3670051838 wait complete. 2024-04-23T18:25:17.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317396 5.11 MB/sec execute 82 sec latency 11661.998 ms 2024-04-23T18:25:18.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:17 smithi172 ceph-mon[32456]: pgmap v149: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-04-23T18:25:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:17 smithi173 ceph-mon[32906]: pgmap v149: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-04-23T18:25:18.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:17 smithi037 ceph-mon[29404]: pgmap v149: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-04-23T18:25:18.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 319625 5.21 MB/sec execute 83 sec latency 15.482 ms 2024-04-23T18:25:19.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 320938 5.21 MB/sec execute 84 sec latency 149.752 ms 2024-04-23T18:25:20.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:19 smithi172 ceph-mon[32456]: pgmap v150: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-04-23T18:25:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:19 smithi173 ceph-mon[32906]: pgmap v150: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-04-23T18:25:20.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:19 smithi037 ceph-mon[29404]: pgmap v150: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-04-23T18:25:20.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 320938 5.15 MB/sec execute 85 sec latency 1149.860 ms 2024-04-23T18:25:21.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 320938 5.09 MB/sec execute 86 sec latency 2149.917 ms 2024-04-23T18:25:22.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:21 smithi172 ceph-mon[32456]: pgmap v151: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-04-23T18:25:22.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:21 smithi173 ceph-mon[32906]: pgmap v151: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-04-23T18:25:22.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:21 smithi037 ceph-mon[29404]: pgmap v151: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.5 MiB/s wr, 83 op/s 2024-04-23T18:25:22.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 320938 5.03 MB/sec execute 87 sec latency 3150.017 ms 2024-04-23T18:25:23.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 321648 5.00 MB/sec execute 88 sec latency 3887.312 ms 2024-04-23T18:25:24.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:23 smithi172 ceph-mon[32456]: pgmap v152: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-23T18:25:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:23 smithi173 ceph-mon[32906]: pgmap v152: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-23T18:25:24.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:23 smithi037 ceph-mon[29404]: pgmap v152: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-04-23T18:25:24.819 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323352 5.08 MB/sec execute 89 sec latency 19.210 ms 2024-04-23T18:25:25.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 324487 5.08 MB/sec execute 90 sec latency 273.105 ms 2024-04-23T18:25:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:25 smithi172 ceph-mon[32456]: pgmap v153: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-04-23T18:25:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:25 smithi173 ceph-mon[32906]: pgmap v153: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-04-23T18:25:26.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:25 smithi037 ceph-mon[29404]: pgmap v153: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-04-23T18:25:26.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 324487 5.02 MB/sec execute 91 sec latency 1273.212 ms 2024-04-23T18:25:27.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 325190 4.99 MB/sec execute 92 sec latency 2039.108 ms 2024-04-23T18:25:28.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:27 smithi172 ceph-mon[32456]: pgmap v154: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 170 op/s 2024-04-23T18:25:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:27 smithi173 ceph-mon[32906]: pgmap v154: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 170 op/s 2024-04-23T18:25:28.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:27 smithi037 ceph-mon[29404]: pgmap v154: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 170 op/s 2024-04-23T18:25:28.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 326786 5.06 MB/sec execute 93 sec latency 19.008 ms 2024-04-23T18:25:29.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 328029 5.06 MB/sec execute 94 sec latency 120.036 ms 2024-04-23T18:25:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:29 smithi172 ceph-mon[32456]: pgmap v155: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-23T18:25:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:29 smithi173 ceph-mon[32906]: pgmap v155: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-23T18:25:30.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:29 smithi037 ceph-mon[29404]: pgmap v155: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 -- 172.21.15.37:0/762730106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 msgr2=0x7fee4810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 --2- 172.21.15.37:0/762730106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee4810dd40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fee40009ca0 tx=0x7fee4002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 -- 172.21.15.37:0/762730106 shutdown_connections 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 --2- 172.21.15.37:0/762730106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee4810dd40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 --2- 172.21.15.37:0/762730106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee48101aa0 0x7fee48105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 --2- 172.21.15.37:0/762730106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee481010f0 0x7fee481014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 -- 172.21.15.37:0/762730106 >> 172.21.15.37:0/762730106 conn(0x7fee48077fd0 msgr2=0x7fee480783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 -- 172.21.15.37:0/762730106 shutdown_connections 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.716+0000 7fee4cd43640 1 -- 172.21.15.37:0/762730106 wait complete. 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 Processor -- start 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 -- start start 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee481010f0 0x7fee481a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee481a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee4819a6f0 0x7fee4819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee481110e0 con 0x7fee48106170 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee48110f60 con 0x7fee4819a6f0 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4cd43640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee48111260 con 0x7fee481010f0 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee481a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee481a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44204/0 (socket says 172.21.15.37:44204) 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 -- 172.21.15.37:0/339092000 learned_addr learned my addr 172.21.15.37:0/339092000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4659c640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee481010f0 0x7fee481a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee46d9d640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee4819a6f0 0x7fee4819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 -- 172.21.15.37:0/339092000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee481010f0 msgr2=0x7fee481a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee481010f0 0x7fee481a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 -- 172.21.15.37:0/339092000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee4819a6f0 msgr2=0x7fee4819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee4819a6f0 0x7fee4819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee45d9b640 1 -- 172.21.15.37:0/339092000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4819b2d0 con 0x7fee48106170 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.717+0000 7fee4659c640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee481010f0 0x7fee481a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.718+0000 7fee45d9b640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee481a0ae0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d740 tx=0x7fee3c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.718+0000 7fee377fe640 1 -- 172.21.15.37:0/339092000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c017070 con 0x7fee48106170 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.718+0000 7fee377fe640 1 -- 172.21.15.37:0/339092000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c00de50 con 0x7fee48106170 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.718+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee481a73b0 con 0x7fee48106170 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.718+0000 7fee377fe640 1 -- 172.21.15.37:0/339092000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c004ce0 con 0x7fee48106170 2024-04-23T18:25:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.718+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee481a7760 con 0x7fee48106170 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.720+0000 7fee377fe640 1 -- 172.21.15.37:0/339092000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee3c004e80 con 0x7fee48106170 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 Processor -- start 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 -- start start 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fd5828970 con 0x564fd58c3620 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fd57c54a0 con 0x564fd58ce800 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4fa57740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564fd5824930 con 0x564fd58c3f40 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee377fe640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee300a3dc0 0x7fee300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee46d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee46d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44218/0 (socket says 172.21.15.37:44218) 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee46d9d640 1 -- 172.21.15.37:0/4091936948 learned_addr learned my addr 172.21.15.37:0/4091936948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee377fe640 1 -- 172.21.15.37:0/339092000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fee3c0d0100 con 0x7fee48106170 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee46d9d640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee300a3dc0 0x7fee300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee4659c640 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.721+0000 7fee45d9b640 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 -- 172.21.15.37:0/4091936948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 msgr2=0x564fd58ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 -- 172.21.15.37:0/4091936948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 msgr2=0x564fd58d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 -- 172.21.15.37:0/4091936948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd553b210 con 0x564fd58c3620 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58c3a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fee40033040 tx=0x7fee40031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f536f9be064911 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee46d9d640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee300a3dc0 0x7fee300a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee40031670 tx=0x7fee40043000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee34ff9640 1 -- 172.21.15.37:0/4091936948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4004f070 con 0x564fd58c3620 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee34ff9640 1 -- 172.21.15.37:0/4091936948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee40036040 con 0x564fd58c3620 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.722+0000 7fee34ff9640 1 -- 172.21.15.37:0/4091936948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400046a0 con 0x564fd58c3620 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 -- 172.21.15.37:0/4091936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 msgr2=0x564fd58c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58c3a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fee40033040 tx=0x7fee40031600 comp rx=0 tx=0).stop 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 -- 172.21.15.37:0/4091936948 shutdown_connections 2024-04-23T18:25:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4091936948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58c3a00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 -- 172.21.15.37:0/4091936948 >> 172.21.15.37:0/4091936948 conn(0x564fd58bfb30 msgr2=0x564fd58bff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 -- 172.21.15.37:0/4091936948 shutdown_connections 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 -- 172.21.15.37:0/4091936948 wait complete. 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 -- start start 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.723+0000 7fee4fa57740 1 Processor -- start 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fd58d13d0 con 0x564fd58c3620 2024-04-23T18:25:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fd57c30b0 con 0x564fd58ce800 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564fd58d1550 con 0x564fd58c3f40 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4fa57740 10 client.0 _ll_register_callbacks cb 0x564fd57b0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44232/0 (socket says 172.21.15.37:44232) 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 -- 172.21.15.37:0/2041093772 learned_addr learned my addr 172.21.15.37:0/2041093772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee45d9b640 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 -- 172.21.15.37:0/2041093772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 msgr2=0x564fd58cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 -- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 msgr2=0x564fd58cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 -- 172.21.15.37:0/2041093772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd5607620 con 0x564fd58c3620 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee4659c640 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58ce200 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fee380117c0 tx=0x7fee38011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82a602a6b795c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.724+0000 7fee2dffb640 1 -- 172.21.15.37:0/2041093772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800f9a0 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee2dffb640 1 -- 172.21.15.37:0/2041093772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fee3800fb40 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee40036040 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee2dffb640 1 -- 172.21.15.37:0/2041093772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee3800fd40 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 10 client.16113 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564fd58dd3e0 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee2dffb640 1 -- 172.21.15.37:0/2041093772 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee40036040 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee2dffb640 1 -- 172.21.15.37:0/2041093772 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fee38018030 con 0x564fd58c3620 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 10 client.16113 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 20 client.16113 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd58de8f0 0x564fd58e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 4 client.16113 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.725+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd5836390 con 0x564fd58de8f0 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.726+0000 7fee45d9b640 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd58de8f0 0x564fd58e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.726+0000 7fee45d9b640 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd58de8f0 0x564fd58e0d40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8b6ced4d0cbeca server_cookie=5cfff336db72d8e4 in_seq=0 out_seq=0 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.726+0000 7fee2dffb640 10 client.16113 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee2dffb640 1 -- 172.21.15.37:0/2041093772 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fd5836390 con 0x564fd58de8f0 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee2dffb640 10 client.16113 handle_command_reply: tid=0 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 client.16113 shutdown 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 msgr2=0x564fd58ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58ce200 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fee380117c0 tx=0x7fee38011bc0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 shutdown_connections 2024-04-23T18:25:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd58de8f0 0x564fd58e0d40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3f40 0x564fd58cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee2dffb640 0 client.16113 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 --2- 172.21.15.37:0/2041093772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58ce200 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 >> 172.21.15.37:0/2041093772 conn(0x564fd58bfb30 msgr2=0x564fd58c5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 shutdown_connections 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 1 -- 172.21.15.37:0/2041093772 wait complete. 2024-04-23T18:25:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.727+0000 7fee4fa57740 20 client.16113 trim_cache size 0 max 16384 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 Processor -- start 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 -- start start 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3620 0x564fd58cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 0x564fd58cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee481b2680 con 0x564fd58c3f40 2024-04-23T18:25:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee481afef0 con 0x564fd58ce800 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4fa57740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee481add40 con 0x564fd58c3620 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee45d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 0x564fd58cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee45d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 0x564fd58cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44236/0 (socket says 172.21.15.37:44236) 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee45d9b640 1 -- 172.21.15.37:0/4054998589 learned_addr learned my addr 172.21.15.37:0/4054998589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.742+0000 7fee4659c640 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3620 0x564fd58cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee46d9d640 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee45d9b640 1 -- 172.21.15.37:0/4054998589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3620 msgr2=0x564fd58cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee45d9b640 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3620 0x564fd58cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee45d9b640 1 -- 172.21.15.37:0/4054998589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 msgr2=0x564fd58e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee45d9b640 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58e6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee45d9b640 1 -- 172.21.15.37:0/4054998589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd58cbe30 con 0x564fd58c3f40 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee4659c640 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3620 0x564fd58cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee45d9b640 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 0x564fd58cdd30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009610 tx=0x7fee3c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd85e6ca4dd125a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee2ffff640 1 -- 172.21.15.37:0/4054998589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c020040 con 0x564fd58c3f40 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee2ffff640 1 -- 172.21.15.37:0/4054998589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0a4da0 con 0x564fd58c3f40 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee2ffff640 1 -- 172.21.15.37:0/4054998589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0cdcc0 con 0x564fd58c3f40 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee4fa57740 1 -- 172.21.15.37:0/4054998589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 msgr2=0x564fd58cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.743+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 0x564fd58cdd30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009610 tx=0x7fee3c005ce0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- 172.21.15.37:0/4054998589 shutdown_connections 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58ce800 0x564fd58e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3f40 0x564fd58cdd30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 --2- 172.21.15.37:0/4054998589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58c3620 0x564fd58cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- 172.21.15.37:0/4054998589 >> 172.21.15.37:0/4054998589 conn(0x564fd58bfb30 msgr2=0x564fd58cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- 172.21.15.37:0/4054998589 shutdown_connections 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- 172.21.15.37:0/4054998589 wait complete. 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- start start 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 Processor -- start 2024-04-23T18:25:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58d6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58c3f40 0x564fd58d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58ce800 0x564fd58d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee481ae3b0 con 0x564fd58c3620 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee481af820 con 0x564fd58c3f40 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee481aea80 con 0x564fd58ce800 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4fa57740 10 client.0 _ll_register_callbacks cb 0x7fee38008c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58d6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58d6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44244/0 (socket says 172.21.15.37:44244) 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee4659c640 1 -- 172.21.15.37:0/541305397 learned_addr learned my addr 172.21.15.37:0/541305397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee45d9b640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58c3f40 0x564fd58d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.744+0000 7fee46d9d640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58ce800 0x564fd58d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4659c640 1 -- 172.21.15.37:0/541305397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58ce800 msgr2=0x564fd58d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4659c640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58ce800 0x564fd58d4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4659c640 1 -- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58c3f40 msgr2=0x564fd58d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4659c640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58c3f40 0x564fd58d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4659c640 1 -- 172.21.15.37:0/541305397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd58d4a30 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee45d9b640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58c3f40 0x564fd58d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4659c640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58d6a70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fee38008d50 tx=0x7fee38019b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad5a76a767144d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee2dffb640 1 -- 172.21.15.37:0/541305397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38026070 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee2dffb640 1 -- 172.21.15.37:0/541305397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fee380035f0 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c0a4da0 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee2dffb640 1 -- 172.21.15.37:0/541305397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee3800f790 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4fa57740 10 client.16119 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564fd5944de0 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.745+0000 7fee2dffb640 1 -- 172.21.15.37:0/541305397 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0a4da0 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee2dffb640 1 -- 172.21.15.37:0/541305397 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fee38018070 con 0x564fd58c3620 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee4fa57740 10 client.16119 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee4fa57740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee4fa57740 20 client.16119 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee4fa57740 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd59455b0 0x564fd59479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee4fa57740 4 client.16119 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:25:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fd5830200 con 0x564fd59455b0 2024-04-23T18:25:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee45d9b640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd59455b0 0x564fd59479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee45d9b640 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd59455b0 0x564fd59479d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da65457cf3d87da4 server_cookie=544c5a75705da388 in_seq=0 out_seq=0 2024-04-23T18:25:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.746+0000 7fee2dffb640 10 client.16119 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee2dffb640 1 -- 172.21.15.37:0/541305397 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fd5830200 con 0x564fd59455b0 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee2dffb640 10 client.16119 handle_command_reply: tid=0 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 client.16119 shutdown 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 msgr2=0x564fd58d6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58d6a70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fee38008d50 tx=0x7fee38019b00 comp rx=0 tx=0).stop 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 shutdown_connections 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564fd59455b0 0x564fd59479d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564fd58ce800 0x564fd58d4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fd58c3f40 0x564fd58d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee2dffb640 0 client.16119 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 --2- 172.21.15.37:0/541305397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fd58c3620 0x564fd58d6a70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 >> 172.21.15.37:0/541305397 conn(0x564fd58bfb30 msgr2=0x564fd58be420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 shutdown_connections 2024-04-23T18:25:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 1 -- 172.21.15.37:0/541305397 wait complete. 2024-04-23T18:25:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.795+0000 7fee4fa57740 20 client.16119 trim_cache size 0 max 16384 2024-04-23T18:25:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:25:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:25:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:25:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee300a3dc0 msgr2=0x7fee300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee300a3dc0 0x7fee300a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee40031670 tx=0x7fee40043000 comp rx=0 tx=0).stop 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 msgr2=0x7fee481a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee481a0ae0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d740 tx=0x7fee3c00dc00 comp rx=0 tx=0).stop 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 shutdown_connections 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee300a3dc0 0x7fee300a6280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee4819a6f0 0x7fee4819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee48106170 0x7fee481a0ae0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 --2- 172.21.15.37:0/339092000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee481010f0 0x7fee481a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 >> 172.21.15.37:0/339092000 conn(0x7fee48077fd0 msgr2=0x7fee48069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 shutdown_connections 2024-04-23T18:25:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:30.796+0000 7fee4cd43640 1 -- 172.21.15.37:0/339092000 wait complete. 2024-04-23T18:25:30.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 328164 5.01 MB/sec execute 95 sec latency 1050.839 ms 2024-04-23T18:25:31.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 330162 5.10 MB/sec execute 96 sec latency 38.196 ms 2024-04-23T18:25:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:31 smithi172 ceph-mon[32456]: pgmap v156: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-23T18:25:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:31 smithi173 ceph-mon[32906]: pgmap v156: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-23T18:25:32.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:31 smithi037 ceph-mon[29404]: pgmap v156: 97 pgs: 97 active+clean; 181 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-23T18:25:32.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 331534 5.11 MB/sec execute 97 sec latency 23.143 ms 2024-04-23T18:25:33.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 332303 5.07 MB/sec execute 98 sec latency 695.525 ms 2024-04-23T18:25:34.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:33 smithi172 ceph-mon[32456]: pgmap v157: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-04-23T18:25:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:33 smithi173 ceph-mon[32906]: pgmap v157: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-04-23T18:25:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:33 smithi037 ceph-mon[29404]: pgmap v157: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-04-23T18:25:34.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 332376 5.03 MB/sec execute 99 sec latency 925.747 ms 2024-04-23T18:25:35.820 INFO:tasks.workunit.client.0.smithi037.stdout: 1 332493 4.98 MB/sec execute 100 sec latency 1372.540 ms 2024-04-23T18:25:36.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:35 smithi172 ceph-mon[32456]: pgmap v158: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-04-23T18:25:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:35 smithi173 ceph-mon[32906]: pgmap v158: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-04-23T18:25:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:35 smithi037 ceph-mon[29404]: pgmap v158: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-04-23T18:25:36.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 332493 4.93 MB/sec execute 101 sec latency 1497.600 ms 2024-04-23T18:25:37.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 333373 4.97 MB/sec execute 102 sec latency 2241.892 ms 2024-04-23T18:25:38.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:37 smithi172 ceph-mon[32456]: pgmap v159: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-23T18:25:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:37 smithi173 ceph-mon[32906]: pgmap v159: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-23T18:25:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:37 smithi037 ceph-mon[29404]: pgmap v159: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-23T18:25:38.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 334521 4.96 MB/sec execute 103 sec latency 19.894 ms 2024-04-23T18:25:39.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335120 4.95 MB/sec execute 104 sec latency 575.364 ms 2024-04-23T18:25:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:39 smithi172 ceph-mon[32456]: pgmap v160: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-04-23T18:25:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:39 smithi173 ceph-mon[32906]: pgmap v160: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-04-23T18:25:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:39 smithi037 ceph-mon[29404]: pgmap v160: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-04-23T18:25:40.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335120 4.90 MB/sec execute 105 sec latency 1575.468 ms 2024-04-23T18:25:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:25:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:40 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:25:41.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:25:41.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:41.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:40 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:25:41.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:25:41.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:41.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:41.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:40 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:25:41.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335120 4.85 MB/sec execute 106 sec latency 2575.536 ms 2024-04-23T18:25:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:41 smithi172 ceph-mon[32456]: pgmap v161: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-04-23T18:25:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:41 smithi173 ceph-mon[32906]: pgmap v161: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-04-23T18:25:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:41 smithi037 ceph-mon[29404]: pgmap v161: 97 pgs: 97 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-04-23T18:25:42.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335120 4.81 MB/sec execute 107 sec latency 3575.603 ms 2024-04-23T18:25:43.190 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-23T18:25:43.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:43.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 -- 172.21.15.37:0/994318212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790106190 msgr2=0x7f279010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 --2- 172.21.15.37:0/994318212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790106190 0x7f279010dd60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ca0 tx=0x7f278802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 -- 172.21.15.37:0/994318212 shutdown_connections 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 --2- 172.21.15.37:0/994318212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790106190 0x7f279010dd60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 --2- 172.21.15.37:0/994318212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790101ac0 0x7f2790105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 --2- 172.21.15.37:0/994318212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2790101110 0x7f27901014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 -- 172.21.15.37:0/994318212 >> 172.21.15.37:0/994318212 conn(0x7f2790077fb0 msgr2=0x7f27900783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 -- 172.21.15.37:0/994318212 shutdown_connections 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.314+0000 7f2795eae640 1 -- 172.21.15.37:0/994318212 wait complete. 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 Processor -- start 2024-04-23T18:25:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 -- start start 2024-04-23T18:25:43.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 0x7f27901995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2790106190 0x7f2790199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279019a020 0x7f27901a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111100 con 0x7f2790101110 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2790110f80 con 0x7f279019a020 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f2795eae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111280 con 0x7f2790106190 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 0x7f27901995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.315+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 0x7f27901995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37280/0 (socket says 172.21.15.37:37280) 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 -- 172.21.15.37:0/2125737211 learned_addr learned my addr 172.21.15.37:0/2125737211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278effd640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2790106190 0x7f2790199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278ffff640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279019a020 0x7f27901a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 -- 172.21.15.37:0/2125737211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2790106190 msgr2=0x7f2790199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2790106190 0x7f2790199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 -- 172.21.15.37:0/2125737211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279019a020 msgr2=0x7f27901a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279019a020 0x7f27901a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 -- 172.21.15.37:0/2125737211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27901a4dc0 con 0x7f2790101110 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278f7fe640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 0x7f27901995a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f278000e9f0 tx=0x7f278000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000cd80 con 0x7f2790101110 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27901a50b0 con 0x7f2790101110 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27800054d0 con 0x7f2790101110 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780010690 con 0x7f2790101110 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.316+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a5600 con 0x7f2790101110 2024-04-23T18:25:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.317+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2790102590 con 0x7f2790101110 2024-04-23T18:25:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.319+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2780004e70 con 0x7f2790101110 2024-04-23T18:25:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.319+0000 7f278cff9640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27780a3d50 0x7f27780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.319+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f27800d0ca0 con 0x7f2790101110 2024-04-23T18:25:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.319+0000 7f278effd640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27780a3d50 0x7f27780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.320+0000 7f278effd640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27780a3d50 0x7f27780a6210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f279006b0d0 tx=0x7f278400b410 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.324+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f278009d330 con 0x7f2790101110 2024-04-23T18:25:43.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.551+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2790109420 con 0x7f2790101110 2024-04-23T18:25:43.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.552+0000 7f278cff9640 1 -- 172.21.15.37:0/2125737211 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11881 (secure 0 0 0) 0x7f278009cc30 con 0x7f2790101110 2024-04-23T18:25:43.545 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:25:43.545 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:24:43.883083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:25:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 32 2024-04-23T18:25:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27780a3d50 msgr2=0x7f27780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27780a3d50 0x7f27780a6210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f279006b0d0 tx=0x7f278400b410 comp rx=0 tx=0).stop 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 msgr2=0x7f27901995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 0x7f27901995a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f278000e9f0 tx=0x7f278000edf0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 shutdown_connections 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f27780a3d50 0x7f27780a6210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f279019a020 0x7f27901a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2790106190 0x7f2790199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 --2- 172.21.15.37:0/2125737211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790101110 0x7f27901995a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 >> 172.21.15.37:0/2125737211 conn(0x7f2790077fb0 msgr2=0x7f27900fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 shutdown_connections 2024-04-23T18:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.554+0000 7f2795eae640 1 -- 172.21.15.37:0/2125737211 wait complete. 2024-04-23T18:25:43.558 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-23T18:25:43.558 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 -- 172.21.15.37:0/3683826512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc514107280 msgr2=0x7fc51410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 --2- 172.21.15.37:0/3683826512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc514107280 0x7fc51410ee50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009ca0 tx=0x7fc50c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 -- 172.21.15.37:0/3683826512 shutdown_connections 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 --2- 172.21.15.37:0/3683826512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc514107280 0x7fc51410ee50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 --2- 172.21.15.37:0/3683826512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc514102bf0 0x7fc514106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 --2- 172.21.15.37:0/3683826512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc514102240 0x7fc514102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 -- 172.21.15.37:0/3683826512 >> 172.21.15.37:0/3683826512 conn(0x7fc5140790e0 msgr2=0x7fc5140794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 -- 172.21.15.37:0/3683826512 shutdown_connections 2024-04-23T18:25:43.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.687+0000 7fc5197ba640 1 -- 172.21.15.37:0/3683826512 wait complete. 2024-04-23T18:25:43.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 Processor -- start 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 -- start start 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc514102240 0x7fc5141986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc514107280 0x7fc514198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 0x7fc51419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141121e0 con 0x7fc51419cfa0 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc514112060 con 0x7fc514107280 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5197ba640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc514112360 con 0x7fc514102240 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 0x7fc51419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 0x7fc51419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37290/0 (socket says 172.21.15.37:37290) 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 -- 172.21.15.37:0/2622352804 learned_addr learned my addr 172.21.15.37:0/2622352804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc512ffd640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc514102240 0x7fc5141986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5127fc640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc514107280 0x7fc514198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 -- 172.21.15.37:0/2622352804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc514102240 msgr2=0x7fc5141986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc514102240 0x7fc5141986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 -- 172.21.15.37:0/2622352804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc514107280 msgr2=0x7fc514198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc514107280 0x7fc514198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.688+0000 7fc5137fe640 1 -- 172.21.15.37:0/2622352804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51419da90 con 0x7fc51419cfa0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.689+0000 7fc5137fe640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 0x7fc51419d450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0098d0 tx=0x7fc50c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.689+0000 7fc4fbfff640 1 -- 172.21.15.37:0/2622352804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003b70 con 0x7fc51419cfa0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.689+0000 7fc4fbfff640 1 -- 172.21.15.37:0/2622352804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc50c003d10 con 0x7fc51419cfa0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.689+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51419dd20 con 0x7fc51419cfa0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.689+0000 7fc4fbfff640 1 -- 172.21.15.37:0/2622352804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c02fa20 con 0x7fc51419cfa0 2024-04-23T18:25:43.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.689+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141a5540 con 0x7fc51419cfa0 2024-04-23T18:25:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.691+0000 7fc4fbfff640 1 -- 172.21.15.37:0/2622352804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc50c02fc40 con 0x7fc51419cfa0 2024-04-23T18:25:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.691+0000 7fc51c5a8740 1 Processor -- start 2024-04-23T18:25:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.691+0000 7fc51c5a8740 1 -- start start 2024-04-23T18:25:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f7819ff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a09b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a0c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f781964370 con 0x55f7819ff9c0 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f7819012e0 con 0x55f781a0a090 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f781960330 con 0x55f7819ff0a0 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a09b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a09b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37302/0 (socket says 172.21.15.37:37302) 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 -- 172.21.15.37:0/506758638 learned_addr learned my addr 172.21.15.37:0/506758638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc5127fc640 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a0c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc5127fc640 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f7819ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc4fbfff640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc4fc0a3d20 0x7fc4fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 -- 172.21.15.37:0/506758638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 msgr2=0x55f7819ff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f7819ff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 -- 172.21.15.37:0/506758638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 msgr2=0x55f781a0c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a0c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc5137fe640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc4fc0a3d20 0x7fc4fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 -- 172.21.15.37:0/506758638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f781742d80 con 0x55f7819ff9c0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc4fbfff640 1 -- 172.21.15.37:0/2622352804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc50c0f1b30 con 0x7fc51419cfa0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.692+0000 7fc512ffd640 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a09b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc504009cf0 tx=0x7fc504011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5346c1614aecf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.693+0000 7fc4f97fa640 1 -- 172.21.15.37:0/506758638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50401e070 con 0x55f7819ff9c0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.693+0000 7fc4f97fa640 1 -- 172.21.15.37:0/506758638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc504009de0 con 0x55f7819ff9c0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.693+0000 7fc4f97fa640 1 -- 172.21.15.37:0/506758638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504019450 con 0x55f7819ff9c0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.693+0000 7fc51c5a8740 1 -- 172.21.15.37:0/506758638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 msgr2=0x55f781a09b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.693+0000 7fc5137fe640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc4fc0a3d20 0x7fc4fc0a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ef430 tx=0x7fc50c0906c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.693+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a09b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc504009cf0 tx=0x7fc504011810 comp rx=0 tx=0).stop 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- 172.21.15.37:0/506758638 shutdown_connections 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a0c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a09b50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/506758638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f7819ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- 172.21.15.37:0/506758638 >> 172.21.15.37:0/506758638 conn(0x55f7819fb3c0 msgr2=0x55f7819fb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- 172.21.15.37:0/506758638 shutdown_connections 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- 172.21.15.37:0/506758638 wait complete. 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- start start 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 Processor -- start 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a07000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a07540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a07a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f781a0ce80 con 0x55f7819ff9c0 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f7818fefb0 con 0x55f781a0a090 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.694+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f781a0d000 con 0x55f7819ff0a0 2024-04-23T18:25:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc51c5a8740 10 client.0 _ll_register_callbacks cb 0x55f7818fe9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a07540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a07540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37318/0 (socket says 172.21.15.37:37318) 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 -- 172.21.15.37:0/3022527993 learned_addr learned my addr 172.21.15.37:0/3022527993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a07a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a07000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 -- 172.21.15.37:0/3022527993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 msgr2=0x55f781a07000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a07000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 -- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 msgr2=0x55f781a07a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a07a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 -- 172.21.15.37:0/3022527993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7817a3510 con 0x55f7819ff9c0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc512ffd640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a07540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc504003c60 tx=0x7fc504003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa973710b11f481 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc4f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/3022527993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50401e040 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/3022527993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc504004030 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc504009de0 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/3022527993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc504030050 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc51c5a8740 10 client.16137 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.695+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f781636670 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/3022527993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc504009de0 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/3022527993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc504021070 con 0x55f7819ff9c0 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc51c5a8740 10 client.16137 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc51c5a8740 10 client.16137 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc51c5a8740 20 client.16137 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a1a400 0x55f781a1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc51c5a8740 4 client.16137 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f781971d90 con 0x55f781a1a400 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a1a400 0x55f781a1c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a1a400 0x55f781a1c7f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54adf09efc6a486b server_cookie=3c9fb422bf58bc9a in_seq=0 out_seq=0 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.696+0000 7fc4f6ff5640 10 client.16137 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/3022527993 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f781971d90 con 0x55f781a1a400 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc4f6ff5640 10 client.16137 handle_command_reply: tid=0 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 client.16137 shutdown 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 msgr2=0x55f781a07540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a07540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc504003c60 tx=0x7fc504003a20 comp rx=0 tx=0).stop 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 shutdown_connections 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a1a400 0x55f781a1c7f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a07a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc4f6ff5640 0 client.16137 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a07540 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3022527993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a07000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 >> 172.21.15.37:0/3022527993 conn(0x55f7819fb3c0 msgr2=0x55f7819fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 shutdown_connections 2024-04-23T18:25:43.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.697+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3022527993 wait complete. 2024-04-23T18:25:43.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.698+0000 7fc51c5a8740 20 client.16137 trim_cache size 0 max 16384 2024-04-23T18:25:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 Processor -- start 2024-04-23T18:25:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 -- start start 2024-04-23T18:25:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 0x55f781a15d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7819ff9c0 0x55f7819fad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f781a0a090 0x55f781a21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b04c0 con 0x55f7819ff0a0 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141add30 con 0x55f7819ff9c0 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141abb80 con 0x55f781a0a090 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc5127fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 0x55f781a15d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc5127fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 0x55f781a15d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37332/0 (socket says 172.21.15.37:37332) 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc5127fc640 1 -- 172.21.15.37:0/3709683879 learned_addr learned my addr 172.21.15.37:0/3709683879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc512ffd640 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7819ff9c0 0x55f7819fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.716+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f781a0a090 0x55f781a21ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc5127fc640 1 -- 172.21.15.37:0/3709683879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f781a0a090 msgr2=0x55f781a21ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f781a0a090 0x55f781a21ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc5127fc640 1 -- 172.21.15.37:0/3709683879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7819ff9c0 msgr2=0x55f7819fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7819ff9c0 0x55f7819fad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc5127fc640 1 -- 172.21.15.37:0/3709683879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f781a1cb60 con 0x55f7819ff0a0 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc512ffd640 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7819ff9c0 0x55f7819fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc5127fc640 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 0x55f781a15d70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc514101b80 tx=0x7fc50800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591dad4c72439f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc4f8ff9640 1 -- 172.21.15.37:0/3709683879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508026e60 con 0x55f7819ff0a0 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc4f8ff9640 1 -- 172.21.15.37:0/3709683879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5080054d0 con 0x55f7819ff0a0 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.717+0000 7fc4f8ff9640 1 -- 172.21.15.37:0/3709683879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5080035f0 con 0x55f7819ff0a0 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3709683879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 msgr2=0x55f781a15d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 0x55f781a15d70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc514101b80 tx=0x7fc50800b390 comp rx=0 tx=0).stop 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3709683879 shutdown_connections 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f781a0a090 0x55f781a21ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7819ff9c0 0x55f7819fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/3709683879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff0a0 0x55f781a15d70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3709683879 >> 172.21.15.37:0/3709683879 conn(0x55f7819fb3c0 msgr2=0x55f781a090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3709683879 shutdown_connections 2024-04-23T18:25:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- 172.21.15.37:0/3709683879 wait complete. 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- start start 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 Processor -- start 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a03cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a04230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a04770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141ac1f0 con 0x55f7819ff9c0 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141ad660 con 0x55f781a0a090 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.718+0000 7fc51c5a8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141ac8c0 con 0x55f7819ff0a0 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc51c5a8740 10 client.0 _ll_register_callbacks cb 0x7fc50800bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a04230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a04230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37338/0 (socket says 172.21.15.37:37338) 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 -- 172.21.15.37:0/2886815894 learned_addr learned my addr 172.21.15.37:0/2886815894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc5127fc640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a03cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc5127fc640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a04770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 -- 172.21.15.37:0/2886815894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 msgr2=0x55f781a03cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a03cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 -- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 msgr2=0x55f781a04770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a04770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 -- 172.21.15.37:0/2886815894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f781a7f860 con 0x55f7819ff9c0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc5127fc640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a03cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc512ffd640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a04230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc504003ab0 tx=0x7fc504009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f276e0d4f0041 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc4f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/2886815894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50401e040 con 0x55f7819ff9c0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/2886815894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc504004320 con 0x55f7819ff9c0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5080054d0 con 0x55f7819ff9c0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/2886815894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc504042050 con 0x55f7819ff9c0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc51c5a8740 10 client.16143 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.719+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f781a80af0 con 0x55f7819ff9c0 2024-04-23T18:25:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/2886815894 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5080054d0 con 0x55f7819ff9c0 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/2886815894 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc504021040 con 0x55f7819ff9c0 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc51c5a8740 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc51c5a8740 10 client.16143 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc51c5a8740 20 client.16143 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a81360 0x55f781a83780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc51c5a8740 4 client.16143 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x55f78196bc00 con 0x55f781a81360 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc5127fc640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a81360 0x55f781a83780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc4f6ff5640 10 client.16143 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.720+0000 7fc5127fc640 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a81360 0x55f781a83780 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10c4844368b1693 server_cookie=a3f2a7cb1004170a in_seq=0 out_seq=0 2024-04-23T18:25:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.792+0000 7fc4f6ff5640 1 -- 172.21.15.37:0/2886815894 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f78196bc00 con 0x55f781a81360 2024-04-23T18:25:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.792+0000 7fc4f6ff5640 10 client.16143 handle_command_reply: tid=0 2024-04-23T18:25:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.792+0000 7fc51c5a8740 1 client.16143 shutdown 2024-04-23T18:25:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 msgr2=0x55f781a04230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a04230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc504003ab0 tx=0x7fc504009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 shutdown_connections 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f781a81360 0x55f781a83780 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f781a0a090 0x55f781a04770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7819ff9c0 0x55f781a04230 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc4f6ff5640 0 client.16143 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 --2- 172.21.15.37:0/2886815894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f7819ff0a0 0x55f781a03cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 >> 172.21.15.37:0/2886815894 conn(0x55f7819fb3c0 msgr2=0x55f781a08830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 shutdown_connections 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 1 -- 172.21.15.37:0/2886815894 wait complete. 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc51c5a8740 20 client.16143 trim_cache size 0 max 16384 2024-04-23T18:25:43.787 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 32, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 18, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "acdfa4cf": { 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 18, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 70.0, 2024-04-23T18:25:43.788 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:43.789 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:25:43.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc4fc0a3d20 msgr2=0x7fc4fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc5197ba640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc4fc0a3d20 0x7fc4fc0a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ef430 tx=0x7fc50c0906c0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 msgr2=0x7fc51419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.793+0000 7fc5197ba640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 0x7fc51419d450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0098d0 tx=0x7fc50c0317b0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 shutdown_connections 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc4fc0a3d20 0x7fc4fc0a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51419cfa0 0x7fc51419d450 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc514107280 0x7fc514198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 --2- 172.21.15.37:0/2622352804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc514102240 0x7fc5141986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 >> 172.21.15.37:0/2622352804 conn(0x7fc5140790e0 msgr2=0x7fc5141001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 shutdown_connections 2024-04-23T18:25:43.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:43.794+0000 7fc5197ba640 1 -- 172.21.15.37:0/2622352804 wait complete. 2024-04-23T18:25:43.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'acdfa4cf', quiesce took 0.6 seconds. Will release after: 10 2024-04-23T18:25:43.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.78 MB/sec execute 108 sec latency 4297.889 ms 2024-04-23T18:25:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 -- 172.21.15.37:0/260849107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 msgr2=0x7ff96410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 --2- 172.21.15.37:0/260849107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff96410ee70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff954009ca0 tx=0x7ff95402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 -- 172.21.15.37:0/260849107 shutdown_connections 2024-04-23T18:25:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 --2- 172.21.15.37:0/260849107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff96410ee70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 --2- 172.21.15.37:0/260849107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff964102b80 0x7ff964106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 --2- 172.21.15.37:0/260849107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 0x7ff9641025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.886+0000 7ff96be59640 1 -- 172.21.15.37:0/260849107 >> 172.21.15.37:0/260849107 conn(0x7ff9640793e0 msgr2=0x7ff9640797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 -- 172.21.15.37:0/260849107 shutdown_connections 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 -- 172.21.15.37:0/260849107 wait complete. 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 Processor -- start 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 -- start start 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 0x7ff964198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff964198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff96419cef0 0x7ff96419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641121f0 con 0x7ff9641072a0 2024-04-23T18:25:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112070 con 0x7ff9641021d0 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff96be59640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff964112370 con 0x7ff96419cef0 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.887+0000 7ff9693f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff964198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff964198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37368/0 (socket says 172.21.15.37:37368) 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 -- 172.21.15.37:0/1586075155 learned_addr learned my addr 172.21.15.37:0/1586075155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff96419cef0 0x7ff96419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff969bf5640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 0x7ff964198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 -- 172.21.15.37:0/1586075155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff96419cef0 msgr2=0x7ff96419d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff96419cef0 0x7ff96419d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 -- 172.21.15.37:0/1586075155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 msgr2=0x7ff964198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 0x7ff964198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 -- 172.21.15.37:0/1586075155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96419d9e0 con 0x7ff9641072a0 2024-04-23T18:25:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff969bf5640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 0x7ff964198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff9693f4640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff964198b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff95800d770 tx=0x7ff95800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff962ffd640 1 -- 172.21.15.37:0/1586075155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958017070 con 0x7ff9641072a0 2024-04-23T18:25:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96419dcd0 con 0x7ff9641072a0 2024-04-23T18:25:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff962ffd640 1 -- 172.21.15.37:0/1586075155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff958009c30 con 0x7ff9641072a0 2024-04-23T18:25:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff962ffd640 1 -- 172.21.15.37:0/1586075155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95800f620 con 0x7ff9641072a0 2024-04-23T18:25:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.888+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641a54c0 con 0x7ff9641072a0 2024-04-23T18:25:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff962ffd640 1 -- 172.21.15.37:0/1586075155 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff95801a070 con 0x7ff9641072a0 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 Processor -- start 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 -- start start 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3f40 0x5555860ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860ae800 0x5555860b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555586008970 con 0x5555860a3620 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555585fa54a0 con 0x5555860ae800 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555586004930 con 0x5555860a3f40 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.891+0000 7ff96a3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37370/0 (socket says 172.21.15.37:37370) 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2205108831 learned_addr learned my addr 172.21.15.37:0/2205108831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff9693f4640 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860ae800 0x5555860b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3f40 0x5555860ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff962ffd640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff94c0a3dc0 0x7ff94c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2205108831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3f40 msgr2=0x5555860ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3f40 0x5555860ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2205108831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860ae800 msgr2=0x5555860b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860ae800 0x5555860b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2205108831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555585d1b210 con 0x5555860a3620 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff94c0a3dc0 0x7ff94c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff962ffd640 1 -- 172.21.15.37:0/1586075155 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff9580d02e0 con 0x7ff9641072a0 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860a3a00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff95402f3e0 tx=0x7ff95402f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb7bf7a521ac3c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff94bfff640 1 -- 172.21.15.37:0/2205108831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954042040 con 0x5555860a3620 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff94c0a3dc0 0x7ff94c0a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff95404d000 tx=0x7ff95404e000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff94bfff640 1 -- 172.21.15.37:0/2205108831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff954036070 con 0x5555860a3620 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.892+0000 7ff94bfff640 1 -- 172.21.15.37:0/2205108831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95402f660 con 0x5555860a3620 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2205108831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 msgr2=0x5555860a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860a3a00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff95402f3e0 tx=0x7ff95402f410 comp rx=0 tx=0).stop 2024-04-23T18:25:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2205108831 shutdown_connections 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860ae800 0x5555860b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3f40 0x5555860ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2205108831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860a3a00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2205108831 >> 172.21.15.37:0/2205108831 conn(0x55558609fb30 msgr2=0x55558609ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2205108831 shutdown_connections 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.893+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2205108831 wait complete. 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 -- start start 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 Processor -- start 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x5555860ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555860b1430 con 0x5555860ae800 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555585fa30b0 con 0x5555860a3f40 2024-04-23T18:25:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5555860b15b0 con 0x5555860a3620 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96eb6e740 10 client.0 _ll_register_callbacks cb 0x555585f90d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37372/0 (socket says 172.21.15.37:37372) 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2869589712 learned_addr learned my addr 172.21.15.37:0/2869589712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff9693f4640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x5555860ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2869589712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 msgr2=0x5555860ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x5555860ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 msgr2=0x5555860ab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860ab420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2869589712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555585de7620 con 0x5555860ae800 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.894+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x5555860ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860ab960 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff95402ade0 tx=0x7ff9540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b3bf1e82c48e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff9497fa640 1 -- 172.21.15.37:0/2869589712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954058070 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff9497fa640 1 -- 172.21.15.37:0/2869589712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff954003970 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff954036070 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff9497fa640 1 -- 172.21.15.37:0/2869589712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff954064050 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 10 client.16155 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5555860b1430 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff9497fa640 1 -- 172.21.15.37:0/2869589712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff954036070 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff9497fa640 1 -- 172.21.15.37:0/2869589712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff9540306c0 con 0x5555860ae800 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 10 client.16155 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 20 client.16155 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555860be4c0 0x5555860c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 4 client.16155 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.895+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555586016390 con 0x5555860be4c0 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.896+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555860be4c0 0x5555860c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.896+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555860be4c0 0x5555860c08b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3564f2da17747b15 server_cookie=9249929e6f4ddeea in_seq=0 out_seq=0 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.896+0000 7ff9497fa640 10 client.16155 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff9497fa640 1 -- 172.21.15.37:0/2869589712 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555586016390 con 0x5555860be4c0 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff9497fa640 10 client.16155 handle_command_reply: tid=0 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 client.16155 shutdown 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 msgr2=0x5555860ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860ab960 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff95402ade0 tx=0x7ff9540095c0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 shutdown_connections 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5555860be4c0 0x5555860c08b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860ab960 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff9497fa640 0 client.16155 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2869589712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x5555860ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 >> 172.21.15.37:0/2869589712 conn(0x55558609fb30 msgr2=0x5555860a5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 shutdown_connections 2024-04-23T18:25:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2869589712 wait complete. 2024-04-23T18:25:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.897+0000 7ff96eb6e740 20 client.16155 trim_cache size 0 max 16384 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 Processor -- start 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 -- start start 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860ae800 0x5555860adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641b03a0 con 0x5555860a3620 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641adc10 con 0x5555860a3f40 2024-04-23T18:25:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641aba60 con 0x5555860ae800 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860ba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37376/0 (socket says 172.21.15.37:37376) 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 -- 172.21.15.37:0/1222418087 learned_addr learned my addr 172.21.15.37:0/1222418087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff9693f4640 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860ae800 0x5555860adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 -- 172.21.15.37:0/1222418087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860ae800 msgr2=0x5555860adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860ae800 0x5555860adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 -- 172.21.15.37:0/1222418087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 msgr2=0x5555860a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff969bf5640 1 -- 172.21.15.37:0/1222418087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555860a3d70 con 0x5555860a3620 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.912+0000 7ff9693f4640 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff969bf5640 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860ba2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7f2a35e35c368b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff94b7fe640 1 -- 172.21.15.37:0/1222418087 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c014b50 con 0x5555860a3620 2024-04-23T18:25:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff94b7fe640 1 -- 172.21.15.37:0/1222418087 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c021070 con 0x5555860a3620 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff94b7fe640 1 -- 172.21.15.37:0/1222418087 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c01b8c0 con 0x5555860a3620 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 -- 172.21.15.37:0/1222418087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 msgr2=0x5555860ba2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860ba2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c00b5a0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 -- 172.21.15.37:0/1222418087 shutdown_connections 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860ae800 0x5555860adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x5555860a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/1222418087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860a3620 0x5555860ba2f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 -- 172.21.15.37:0/1222418087 >> 172.21.15.37:0/1222418087 conn(0x55558609fb30 msgr2=0x5555860a6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 -- 172.21.15.37:0/1222418087 shutdown_connections 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.913+0000 7ff96eb6e740 1 -- 172.21.15.37:0/1222418087 wait complete. 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 -- start start 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 Processor -- start 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x55558611cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x55558611d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641ac0d0 con 0x5555860ae800 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641ad540 con 0x5555860a3f40 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641ac7a0 con 0x5555860a3620 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96eb6e740 10 client.0 _ll_register_callbacks cb 0x7ff95402f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37392/0 (socket says 172.21.15.37:37392) 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2470876072 learned_addr learned my addr 172.21.15.37:0/2470876072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff9693f4640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x55558611d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x55558611cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2470876072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 msgr2=0x55558611cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x55558611cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 msgr2=0x55558611d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x55558611d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.914+0000 7ff96a3f6640 1 -- 172.21.15.37:0/2470876072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555860b2220 con 0x5555860ae800 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x55558611cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96a3f6640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860b1b30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff954001220 tx=0x7ff954031d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19fbc425585da3c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff9497fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff9497fa640 1 -- 172.21.15.37:0/2470876072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954058040 con 0x5555860ae800 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff9497fa640 1 -- 172.21.15.37:0/2470876072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff9540305c0 con 0x5555860ae800 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c021070 con 0x5555860ae800 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff9497fa640 1 -- 172.21.15.37:0/2470876072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff954042040 con 0x5555860ae800 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 10 client.16161 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555586124400 con 0x5555860ae800 2024-04-23T18:25:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff9497fa640 1 -- 172.21.15.37:0/2470876072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c021070 con 0x5555860ae800 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff9497fa640 1 -- 172.21.15.37:0/2470876072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff95402fbf0 con 0x5555860ae800 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 10 client.16161 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 20 client.16161 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555586124a50 0x555586126e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 4 client.16161 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.915+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555860abcf0 con 0x555586124a50 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.916+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555586124a50 0x555586126e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.916+0000 7ff969bf5640 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555586124a50 0x555586126e70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6035a1f3c16215d1 server_cookie=ef393bff7042dde0 in_seq=0 out_seq=0 2024-04-23T18:25:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.916+0000 7ff9497fa640 10 client.16161 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.937+0000 7ff9497fa640 1 -- 172.21.15.37:0/2470876072 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555860abcf0 con 0x555586124a50 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.937+0000 7ff9497fa640 10 client.16161 handle_command_reply: tid=0 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.937+0000 7ff96eb6e740 1 client.16161 shutdown 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 msgr2=0x5555860b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860b1b30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff954001220 tx=0x7ff954031d40 comp rx=0 tx=0).stop 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 shutdown_connections 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555586124a50 0x555586126e70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555860ae800 0x5555860b1b30 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555860a3f40 0x55558611d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff9497fa640 0 client.16161 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 --2- 172.21.15.37:0/2470876072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5555860a3620 0x55558611cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 >> 172.21.15.37:0/2470876072 conn(0x55558609fb30 msgr2=0x5555860a5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 shutdown_connections 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 1 -- 172.21.15.37:0/2470876072 wait complete. 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96eb6e740 20 client.16161 trim_cache size 0 max 16384 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:25:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff94c0a3dc0 msgr2=0x7ff94c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96be59640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff94c0a3dc0 0x7ff94c0a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff95404d000 tx=0x7ff95404e000 comp rx=0 tx=0).stop 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 msgr2=0x7ff964198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.938+0000 7ff96be59640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff964198b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff95800d770 tx=0x7ff95800dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 shutdown_connections 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff94c0a3dc0 0x7ff94c0a6280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff96419cef0 0x7ff96419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9641072a0 0x7ff964198b70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 --2- 172.21.15.37:0/1586075155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9641021d0 0x7ff964198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 >> 172.21.15.37:0/1586075155 conn(0x7ff9640793e0 msgr2=0x7ff9640ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 shutdown_connections 2024-04-23T18:25:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:43.939+0000 7ff96be59640 1 -- 172.21.15.37:0/1586075155 wait complete. 2024-04-23T18:25:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:43 smithi172 ceph-mon[32456]: pgmap v162: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-04-23T18:25:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:43 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2125737211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:43 smithi173 ceph-mon[32906]: pgmap v162: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-04-23T18:25:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:43 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2125737211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:43 smithi037 ceph-mon[29404]: pgmap v162: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-04-23T18:25:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:43 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2125737211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:44.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.73 MB/sec execute 109 sec latency 1039.830 ms 2024-04-23T18:25:45.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.69 MB/sec execute 110 sec latency 2039.886 ms 2024-04-23T18:25:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:45 smithi172 ceph-mon[32456]: pgmap v163: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-04-23T18:25:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:45 smithi173 ceph-mon[32906]: pgmap v163: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-04-23T18:25:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:45 smithi037 ceph-mon[29404]: pgmap v163: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2024-04-23T18:25:46.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.65 MB/sec execute 111 sec latency 3039.934 ms 2024-04-23T18:25:47.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.61 MB/sec execute 112 sec latency 4040.017 ms 2024-04-23T18:25:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:47 smithi172 ceph-mon[32456]: pgmap v164: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-04-23T18:25:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:47 smithi173 ceph-mon[32906]: pgmap v164: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-04-23T18:25:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:47 smithi037 ceph-mon[29404]: pgmap v164: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-04-23T18:25:48.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.57 MB/sec execute 113 sec latency 5040.127 ms 2024-04-23T18:25:49.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.53 MB/sec execute 114 sec latency 6040.239 ms 2024-04-23T18:25:50.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:49 smithi172 ceph-mon[32456]: pgmap v165: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 316 KiB/s wr, 67 op/s 2024-04-23T18:25:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:49 smithi173 ceph-mon[32906]: pgmap v165: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 316 KiB/s wr, 67 op/s 2024-04-23T18:25:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:49 smithi037 ceph-mon[29404]: pgmap v165: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 316 KiB/s wr, 67 op/s 2024-04-23T18:25:50.821 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.49 MB/sec execute 115 sec latency 7040.295 ms 2024-04-23T18:25:51.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.45 MB/sec execute 116 sec latency 8040.404 ms 2024-04-23T18:25:52.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:51 smithi172 ceph-mon[32456]: pgmap v166: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 316 KiB/s wr, 67 op/s 2024-04-23T18:25:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:51 smithi173 ceph-mon[32906]: pgmap v166: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 316 KiB/s wr, 67 op/s 2024-04-23T18:25:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:51 smithi037 ceph-mon[29404]: pgmap v166: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 316 KiB/s wr, 67 op/s 2024-04-23T18:25:52.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.41 MB/sec execute 117 sec latency 9040.530 ms 2024-04-23T18:25:53.798 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:53.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 335766 4.37 MB/sec execute 118 sec latency 10040.585 ms 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 -- 172.21.15.37:0/3360696968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 msgr2=0x7f254c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3360696968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c103590 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f253c009900 tx=0x7f253c02a480 comp rx=0 tx=0).stop 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 -- 172.21.15.37:0/3360696968 shutdown_connections 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3360696968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 0x7f254c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3360696968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 0x7f254c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3360696968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c103590 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 -- 172.21.15.37:0/3360696968 >> 172.21.15.37:0/3360696968 conn(0x7f254c0771d0 msgr2=0x7f254c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 -- 172.21.15.37:0/3360696968 shutdown_connections 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.921+0000 7f2552cfc640 1 -- 172.21.15.37:0/3360696968 wait complete. 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 Processor -- start 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 -- start start 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 0x7f254c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 0x7f254c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f254c1121b0 con 0x7f254c1031b0 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f254c112030 con 0x7f254c103ad0 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2552cfc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f254c112330 con 0x7f254c10c580 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57986/0 (socket says 172.21.15.37:57986) 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 -- 172.21.15.37:0/3708783216 learned_addr learned my addr 172.21.15.37:0/3708783216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2551299640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 0x7f254c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 -- 172.21.15.37:0/3708783216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 msgr2=0x7f254c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 0x7f254c19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f254bfff640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 0x7f254c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 -- 172.21.15.37:0/3708783216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 msgr2=0x7f254c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 0x7f254c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 -- 172.21.15.37:0/3708783216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c19d740 con 0x7f254c1031b0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2551299640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 0x7f254c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f254bfff640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 0x7f254c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.922+0000 7f2550a98640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c198750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f253c005940 tx=0x7f253c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.923+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c03f030 con 0x7f254c1031b0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.923+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c003aa0 con 0x7f254c1031b0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.923+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c19d9d0 con 0x7f254c1031b0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.923+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c035070 con 0x7f254c1031b0 2024-04-23T18:25:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.923+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a52c0 con 0x7f254c1031b0 2024-04-23T18:25:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.924+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c06ad70 con 0x7f254c1031b0 2024-04-23T18:25:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.925+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f253c042030 con 0x7f254c1031b0 2024-04-23T18:25:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.926+0000 7f2549ffb640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f25340a3d20 0x7f25340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.926+0000 7f254bfff640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f25340a3d20 0x7f25340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.926+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f253c0f11c0 con 0x7f254c1031b0 2024-04-23T18:25:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.927+0000 7f254bfff640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f25340a3d20 0x7f25340a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f254c10e490 tx=0x7f2540005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:53.930+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f253c0bd850 con 0x7f254c1031b0 2024-04-23T18:25:54.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:53 smithi037 ceph-mon[29404]: pgmap v167: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 320 KiB/s wr, 70 op/s 2024-04-23T18:25:54.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.158+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f254c0691c0 con 0x7f254c1031b0 2024-04-23T18:25:54.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.159+0000 7f2549ffb640 1 -- 172.21.15.37:0/3708783216 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11881 (secure 0 0 0) 0x7f253c0bd150 con 0x7f254c1031b0 2024-04-23T18:25:54.151 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:25:54.152 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:24:43.883083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:25:54.152 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 32 2024-04-23T18:25:54.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f25340a3d20 msgr2=0x7f25340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f25340a3d20 0x7f25340a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f254c10e490 tx=0x7f2540005f90 comp rx=0 tx=0).stop 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 msgr2=0x7f254c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c198750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f253c005940 tx=0x7f253c004270 comp rx=0 tx=0).stop 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 shutdown_connections 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f25340a3d20 0x7f25340a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f254c10c580 0x7f254c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c103ad0 0x7f254c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 --2- 172.21.15.37:0/3708783216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f254c1031b0 0x7f254c198750 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 >> 172.21.15.37:0/3708783216 conn(0x7f254c0771d0 msgr2=0x7f254c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 shutdown_connections 2024-04-23T18:25:54.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.161+0000 7f2552cfc640 1 -- 172.21.15.37:0/3708783216 wait complete. 2024-04-23T18:25:54.164 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id acdfa4cf --release --await' 2024-04-23T18:25:54.164 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id acdfa4cf --release --await 2024-04-23T18:25:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 -- 172.21.15.37:0/3183969238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406aef0 msgr2=0x7fbb9410e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/3183969238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406aef0 0x7fbb9410e420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802a400 comp rx=0 tx=0).stop 2024-04-23T18:25:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 -- 172.21.15.37:0/3183969238 shutdown_connections 2024-04-23T18:25:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/3183969238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb9410e960 0x7fbb94110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/3183969238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406aef0 0x7fbb9410e420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/3183969238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb9406a5d0 0x7fbb9406a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.323+0000 7fbb9c54f640 1 -- 172.21.15.37:0/3183969238 >> 172.21.15.37:0/3183969238 conn(0x7fbb94077eb0 msgr2=0x7fbb940782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.325+0000 7fbb9c54f640 1 -- 172.21.15.37:0/3183969238 shutdown_connections 2024-04-23T18:25:54.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.326+0000 7fbb9c54f640 1 -- 172.21.15.37:0/3183969238 wait complete. 2024-04-23T18:25:54.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.326+0000 7fbb9c54f640 1 Processor -- start 2024-04-23T18:25:54.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.326+0000 7fbb9c54f640 1 -- start start 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9c54f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb9406a5d0 0x7fbb94103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9c54f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb9406aef0 0x7fbb941035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9c54f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 0x7fbb94103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9c54f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114400 con 0x7fbb9410e960 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9c54f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114280 con 0x7fbb9406aef0 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9c54f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114580 con 0x7fbb9406a5d0 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9aaec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 0x7fbb94103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9aaec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 0x7fbb94103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58010/0 (socket says 172.21.15.37:58010) 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb9aaec640 1 -- 172.21.15.37:0/1821880630 learned_addr learned my addr 172.21.15.37:0/1821880630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.327+0000 7fbb99aea640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb9406aef0 0x7fbb941035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9aaec640 1 -- 172.21.15.37:0/1821880630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb9406a5d0 msgr2=0x7fbb94103090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:25:54.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9aaec640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb9406a5d0 0x7fbb94103090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9aaec640 1 -- 172.21.15.37:0/1821880630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb9406aef0 msgr2=0x7fbb941035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9aaec640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb9406aef0 0x7fbb941035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9aaec640 1 -- 172.21.15.37:0/1821880630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941aaf90 con 0x7fbb9410e960 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9aaec640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 0x7fbb94103b10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eb70 tx=0x7fbb8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb937fe640 1 -- 172.21.15.37:0/1821880630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004d20 con 0x7fbb9410e960 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941ab280 con 0x7fbb9410e960 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb937fe640 1 -- 172.21.15.37:0/1821880630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb8c00ce70 con 0x7fbb9410e960 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb937fe640 1 -- 172.21.15.37:0/1821880630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c018760 con 0x7fbb9410e960 2024-04-23T18:25:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.328+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941ab7a0 con 0x7fbb9410e960 2024-04-23T18:25:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.331+0000 7fbb937fe640 1 -- 172.21.15.37:0/1821880630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb8c018900 con 0x7fbb9410e960 2024-04-23T18:25:54.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.331+0000 7fbb9f26b740 1 Processor -- start 2024-04-23T18:25:54.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 -- start start 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555cd554d6d0 con 0x555cd55e8db0 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555cd54ea1a0 con 0x555cd55e8490 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555cd5549690 con 0x555cd55f3480 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb937fe640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9a2eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9a2eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58024/0 (socket says 172.21.15.37:58024) 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/3779332738 learned_addr learned my addr 172.21.15.37:0/3779332738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb99aea640 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb9aaec640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb99aea640 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.332+0000 7fbb937fe640 1 -- 172.21.15.37:0/1821880630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbb8c0d0f20 con 0x7fbb9410e960 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/3779332738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 msgr2=0x555cd55f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/3779332738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 msgr2=0x555cd55e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/3779332738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd54f45b0 con 0x555cd55e8db0 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9aaec640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c099880 tx=0x7fbb8c0c7fc0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55f2f40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009bd0 tx=0x7fbb8400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345dbe521b7db2b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb90bfe640 1 -- 172.21.15.37:0/3779332738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84005020 con 0x555cd55e8db0 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb90bfe640 1 -- 172.21.15.37:0/3779332738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb840047a0 con 0x555cd55e8db0 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.333+0000 7fbb90bfe640 1 -- 172.21.15.37:0/3779332738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84010690 con 0x555cd55e8db0 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 -- 172.21.15.37:0/3779332738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 msgr2=0x555cd55f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55f2f40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009bd0 tx=0x7fbb8400e720 comp rx=0 tx=0).stop 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 -- 172.21.15.37:0/3779332738 shutdown_connections 2024-04-23T18:25:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55f2f40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/3779332738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 -- 172.21.15.37:0/3779332738 >> 172.21.15.37:0/3779332738 conn(0x555cd55e47b0 msgr2=0x555cd55e4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 -- 172.21.15.37:0/3779332738 shutdown_connections 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 -- 172.21.15.37:0/3779332738 wait complete. 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 -- start start 2024-04-23T18:25:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 Processor -- start 2024-04-23T18:25:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.334+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55e8db0 0x555cd55f0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55f3480 0x555cd55f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555cd55f6310 con 0x555cd55e8490 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555cd54e7b20 con 0x555cd55f3480 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555cd55f6490 con 0x555cd55e8db0 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9f26b740 10 client.0 _ll_register_callbacks cb 0x555cd54e7530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58030/0 (socket says 172.21.15.37:58030) 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 -- 172.21.15.37:0/1813866195 learned_addr learned my addr 172.21.15.37:0/1813866195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55e8db0 0x555cd55f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55f3480 0x555cd55f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 -- 172.21.15.37:0/1813866195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55e8db0 msgr2=0x555cd55f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55e8db0 0x555cd55f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 -- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55f3480 msgr2=0x555cd55f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55f3480 0x555cd55f07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 -- 172.21.15.37:0/1813866195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd525f920 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb99aea640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55f2ea0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009230 tx=0x7fbb8802a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4279b3eefe429adf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb79ffb640 1 -- 172.21.15.37:0/1813866195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88035040 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb840047a0 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.335+0000 7fbb79ffb640 1 -- 172.21.15.37:0/1813866195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbb88003d70 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb79ffb640 1 -- 172.21.15.37:0/1813866195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb88053050 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 10 client.16179 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555cd532bd30 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb79ffb640 1 -- 172.21.15.37:0/1813866195 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb840047a0 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb79ffb640 1 -- 172.21.15.37:0/1813866195 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbb8803d450 con 0x555cd55e8490 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 10 client.16179 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 10 client.16179 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 20 client.16179 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd5605a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 4 client.16179 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd550b500 con 0x555cd5603580 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.336+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd5605a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd5605a00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b8ae59ea9ddca7 server_cookie=382c701efa3902e1 in_seq=0 out_seq=0 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb79ffb640 10 client.16179 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb79ffb640 1 -- 172.21.15.37:0/1813866195 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cd550b500 con 0x555cd5603580 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb79ffb640 10 client.16179 handle_command_reply: tid=0 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 client.16179 shutdown 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 msgr2=0x555cd55f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55f2ea0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009230 tx=0x7fbb8802a400 comp rx=0 tx=0).stop 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 shutdown_connections 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd5605a00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55f3480 0x555cd55f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55e8db0 0x555cd55f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb79ffb640 0 client.16179 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/1813866195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55f2ea0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.337+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 >> 172.21.15.37:0/1813866195 conn(0x555cd55e47b0 msgr2=0x555cd55e39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.338+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 shutdown_connections 2024-04-23T18:25:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.338+0000 7fbb9f26b740 1 -- 172.21.15.37:0/1813866195 wait complete. 2024-04-23T18:25:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.338+0000 7fbb9f26b740 20 client.16179 trim_cache size 0 max 16384 2024-04-23T18:25:54.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:53 smithi172 ceph-mon[32456]: pgmap v167: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 320 KiB/s wr, 70 op/s 2024-04-23T18:25:54.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 Processor -- start 2024-04-23T18:25:54.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 -- start start 2024-04-23T18:25:54.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55e4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8db0 0x555cd55f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd560b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941b6b30 con 0x555cd55e8490 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941b43a0 con 0x555cd55e8db0 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941b21f0 con 0x555cd55f3480 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb99aea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb99aea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55e4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58040/0 (socket says 172.21.15.37:58040) 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.355+0000 7fbb99aea640 1 -- 172.21.15.37:0/4061334119 learned_addr learned my addr 172.21.15.37:0/4061334119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8db0 0x555cd55f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb99aea640 1 -- 172.21.15.37:0/4061334119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 msgr2=0x555cd560b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb99aea640 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd560b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb99aea640 1 -- 172.21.15.37:0/4061334119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8db0 msgr2=0x555cd55f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb99aea640 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8db0 0x555cd55f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb99aea640 1 -- 172.21.15.37:0/4061334119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd55e0170 con 0x555cd55e8490 2024-04-23T18:25:54.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8db0 0x555cd55f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb99aea640 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55e4150 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbb8802a920 tx=0x7fbb88031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b483fcf2b89a3d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb7bfff640 1 -- 172.21.15.37:0/4061334119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880040c0 con 0x555cd55e8490 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb7bfff640 1 -- 172.21.15.37:0/4061334119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb880054e0 con 0x555cd55e8490 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb7bfff640 1 -- 172.21.15.37:0/4061334119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88004920 con 0x555cd55e8490 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb9f26b740 1 -- 172.21.15.37:0/4061334119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 msgr2=0x555cd55e4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.356+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55e4150 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbb8802a920 tx=0x7fbb88031400 comp rx=0 tx=0).stop 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- 172.21.15.37:0/4061334119 shutdown_connections 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd560b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8db0 0x555cd55f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/4061334119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8490 0x555cd55e4150 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- 172.21.15.37:0/4061334119 >> 172.21.15.37:0/4061334119 conn(0x555cd55e47b0 msgr2=0x555cd55eb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- 172.21.15.37:0/4061334119 shutdown_connections 2024-04-23T18:25:54.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- 172.21.15.37:0/4061334119 wait complete. 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- start start 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 Processor -- start 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941b2860 con 0x555cd55e8db0 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941b3cd0 con 0x555cd55e8490 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941b2f30 con 0x555cd55f3480 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9f26b740 10 client.0 _ll_register_callbacks cb 0x7fbb88003730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9a2eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9a2eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58050/0 (socket says 172.21.15.37:58050) 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.357+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/417892989 learned_addr learned my addr 172.21.15.37:0/417892989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb99aea640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/417892989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 msgr2=0x555cd55f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb99aea640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 msgr2=0x555cd55edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9a2eb640 1 -- 172.21.15.37:0/417892989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd55f4a90 con 0x555cd55e8db0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb99aea640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9a2eb640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55fb6e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009fd0 tx=0x7fbb84005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d22cf2e9c1604a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb79ffb640 1 -- 172.21.15.37:0/417892989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8402e060 con 0x555cd55e8db0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb79ffb640 1 -- 172.21.15.37:0/417892989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbb84004040 con 0x555cd55e8db0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb880054e0 con 0x555cd55e8db0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb79ffb640 1 -- 172.21.15.37:0/417892989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb8401d040 con 0x555cd55e8db0 2024-04-23T18:25:54.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9f26b740 10 client.16185 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555cd566a1d0 con 0x555cd55e8db0 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb79ffb640 1 -- 172.21.15.37:0/417892989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb880054e0 con 0x555cd55e8db0 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb79ffb640 1 -- 172.21.15.37:0/417892989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbb840047a0 con 0x555cd55e8db0 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9f26b740 10 client.16185 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9f26b740 10 client.16185 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.358+0000 7fbb9f26b740 20 client.16185 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.359+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd566c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.359+0000 7fbb9f26b740 4 client.16185 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "acdfa4cf", "release": true, "await": true}] 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.359+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "acdfa4cf", "release": true, "await": true}) -- 0x555cd555b0f0 con 0x555cd5603580 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.359+0000 7fbb99aea640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd566c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.359+0000 7fbb99aea640 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd566c6b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ece678d26a8cfa9 server_cookie=dec53d55d36a31db in_seq=0 out_seq=0 2024-04-23T18:25:54.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.359+0000 7fbb79ffb640 10 client.16185 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:54.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:53 smithi173 ceph-mon[32906]: pgmap v167: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 320 KiB/s wr, 70 op/s 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb79ffb640 1 -- 172.21.15.37:0/417892989 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555cd555b0f0 con 0x555cd5603580 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb79ffb640 10 client.16185 handle_command_reply: tid=0 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 client.16185 shutdown 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 msgr2=0x555cd55fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55fb6e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009fd0 tx=0x7fbb84005ab0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 shutdown_connections 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555cd5603580 0x555cd566c6b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555cd55f3480 0x555cd55f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb79ffb640 0 client.16185 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cd55e8db0 0x555cd55fb6e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 --2- 172.21.15.37:0/417892989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555cd55e8490 0x555cd55edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 >> 172.21.15.37:0/417892989 conn(0x555cd55e47b0 msgr2=0x555cd55e9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 shutdown_connections 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 1 -- 172.21.15.37:0/417892989 wait complete. 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9f26b740 20 client.16185 trim_cache size 0 max 16384 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 32, 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 20, 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout: "acdfa4cf": { 2024-04-23T18:25:54.360 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 20, 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 70.0, 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:25:54.361 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:25:54.362 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbb7c0a3d20 msgr2=0x7fbb7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c099880 tx=0x7fbb8c0c7fc0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 msgr2=0x7fbb94103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 0x7fbb94103b10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eb70 tx=0x7fbb8c00c6a0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 shutdown_connections 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbb7c0a3d20 0x7fbb7c0a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9410e960 0x7fbb94103b10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb9406aef0 0x7fbb941035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 --2- 172.21.15.37:0/1821880630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb9406a5d0 0x7fbb94103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.364+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 >> 172.21.15.37:0/1821880630 conn(0x7fbb94077eb0 msgr2=0x7fbb9410fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.365+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 shutdown_connections 2024-04-23T18:25:54.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:25:54.365+0000 7fbb9c54f640 1 -- 172.21.15.37:0/1821880630 wait complete. 2024-04-23T18:25:54.368 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'acdfa4cf', seconds elapsed: 11.2 2024-04-23T18:25:54.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 337045 4.43 MB/sec execute 119 sec latency 10582.000 ms 2024-04-23T18:25:55.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:54 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3708783216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:54 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3708783216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:54 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3708783216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338387 4.44 MB/sec execute 120 sec latency 18.384 ms 2024-04-23T18:25:56.237 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:55 smithi172 ceph-mon[32456]: pgmap v168: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-23T18:25:56.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:55 smithi173 ceph-mon[32906]: pgmap v168: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-23T18:25:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:55 smithi037 ceph-mon[29404]: pgmap v168: 97 pgs: 97 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-04-23T18:25:56.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 340781 4.52 MB/sec execute 121 sec latency 13.810 ms 2024-04-23T18:25:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 -- 172.21.15.37:0/3216532574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb4100d80 msgr2=0x7fcfb4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/3216532574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb4100d80 0x7fcfb4101160 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009860 tx=0x7fcfa802a340 comp rx=0 tx=0).stop 2024-04-23T18:25:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 -- 172.21.15.37:0/3216532574 shutdown_connections 2024-04-23T18:25:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/3216532574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfb410e930 0x7fcfb4110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/3216532574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb41016a0 0x7fcfb410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/3216532574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb4100d80 0x7fcfb4101160 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 -- 172.21.15.37:0/3216532574 >> 172.21.15.37:0/3216532574 conn(0x7fcfb4077fd0 msgr2=0x7fcfb40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 -- 172.21.15.37:0/3216532574 shutdown_connections 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.031+0000 7fcfbae6b640 1 -- 172.21.15.37:0/3216532574 wait complete. 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 Processor -- start 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 -- start start 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb4100d80 0x7fcfb41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfb41016a0 0x7fcfb41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 0x7fcfb419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41143e0 con 0x7fcfb410e930 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4114260 con 0x7fcfb4100d80 2024-04-23T18:25:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfbae6b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4114560 con 0x7fcfb41016a0 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 0x7fcfb419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 0x7fcfb419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47190/0 (socket says 172.21.15.37:47190) 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 -- 172.21.15.37:0/784185476 learned_addr learned my addr 172.21.15.37:0/784185476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb4100d80 0x7fcfb41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfb41016a0 0x7fcfb41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 -- 172.21.15.37:0/784185476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfb41016a0 msgr2=0x7fcfb41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfb41016a0 0x7fcfb41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 -- 172.21.15.37:0/784185476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb4100d80 msgr2=0x7fcfb41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb4100d80 0x7fcfb41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.032+0000 7fcfb9408640 1 -- 172.21.15.37:0/784185476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb419af90 con 0x7fcfb410e930 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.033+0000 7fcfb9408640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 0x7fcfb419a7c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00eb40 tx=0x7fcfac00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.033+0000 7fcfb1ffb640 1 -- 172.21.15.37:0/784185476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac00ce40 con 0x7fcfb410e930 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.033+0000 7fcfb1ffb640 1 -- 172.21.15.37:0/784185476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac005020 con 0x7fcfb410e930 2024-04-23T18:25:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.033+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb419b280 con 0x7fcfb410e930 2024-04-23T18:25:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.033+0000 7fcfb1ffb640 1 -- 172.21.15.37:0/784185476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac010650 con 0x7fcfb410e930 2024-04-23T18:25:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.033+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a74a0 con 0x7fcfb410e930 2024-04-23T18:25:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfb1ffb640 1 -- 172.21.15.37:0/784185476 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcfac0054b0 con 0x7fcfb410e930 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 Processor -- start 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 -- start start 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c310fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c311a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c311cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f3c3074970 con 0x55f3c310ff40 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3c30114a0 con 0x55f3c311a800 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f3c3070930 con 0x55f3c310f620 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfb1ffb640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcfa40a3e40 0x7fcfa40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c311a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c310fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c311a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47194/0 (socket says 172.21.15.37:47194) 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.036+0000 7fcfb8c07640 1 -- 172.21.15.37:0/1500133766 learned_addr learned my addr 172.21.15.37:0/1500133766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb1ffb640 1 -- 172.21.15.37:0/784185476 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fcfac0d0de0 con 0x7fcfb410e930 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb9408640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcfa40a3e40 0x7fcfa40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c311cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 -- 172.21.15.37:0/1500133766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 msgr2=0x55f3c311cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c311cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 -- 172.21.15.37:0/1500133766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 msgr2=0x55f3c311a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c311a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 -- 172.21.15.37:0/1500133766 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c2d87210 con 0x55f3c310f620 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c311a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb9408640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcfa40a3e40 0x7fcfa40a6300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ce580 tx=0x7fcfac0057a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c310fa00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009690 tx=0x7fcfa0009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19166973b87132eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcf9effd640 1 -- 172.21.15.37:0/1500133766 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0004050 con 0x55f3c310f620 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcf9effd640 1 -- 172.21.15.37:0/1500133766 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa00041f0 con 0x55f3c310f620 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.037+0000 7fcf9effd640 1 -- 172.21.15.37:0/1500133766 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa001d620 con 0x55f3c310f620 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1500133766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 msgr2=0x55f3c310fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c310fa00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009690 tx=0x7fcfa0009620 comp rx=0 tx=0).stop 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1500133766 shutdown_connections 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c311cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c311a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1500133766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c310fa00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1500133766 >> 172.21.15.37:0/1500133766 conn(0x55f3c310bb30 msgr2=0x55f3c310bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1500133766 shutdown_connections 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1500133766 wait complete. 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 -- start start 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.038+0000 7fcfbdb7f740 1 Processor -- start 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c311a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310ff40 0x55f3c3117450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c31179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f3c311d430 con 0x55f3c311a800 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3c300f0b0 con 0x55f3c310ff40 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f3c311d5b0 con 0x55f3c310f620 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfbdb7f740 10 client.0 _ll_register_callbacks cb 0x55f3c2ffcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c31179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c311a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c311a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:41656/0 (socket says 172.21.15.37:41656) 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 -- 172.21.15.37:0/859195113 learned_addr learned my addr 172.21.15.37:0/859195113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310ff40 0x55f3c3117450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 -- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310ff40 msgr2=0x55f3c3117450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310ff40 0x55f3c3117450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 -- 172.21.15.37:0/859195113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 msgr2=0x55f3c31179b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c31179b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb3fff640 1 -- 172.21.15.37:0/859195113 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c2e53620 con 0x55f3c310f620 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.039+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310ff40 0x55f3c3117450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c311a160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009720 tx=0x7fcfa0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0dcb4697054609 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcf93fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcf93fff640 1 -- 172.21.15.37:0/859195113 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0022070 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcf93fff640 1 -- 172.21.15.37:0/859195113 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fcfa001dc30 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa00041f0 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcf93fff640 1 -- 172.21.15.37:0/859195113 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcfa0028040 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcfbdb7f740 10 client.25726 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3c31293a0 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcf93fff640 1 -- 172.21.15.37:0/859195113 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa00041f0 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.040+0000 7fcf93fff640 1 -- 172.21.15.37:0/859195113 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcfa001e530 con 0x55f3c310f620 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfbdb7f740 10 client.25726 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfbdb7f740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfbdb7f740 20 client.25726 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c312a6f0 0x55f3c312cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfbdb7f740 4 client.25726 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c3082390 con 0x55f3c312a6f0 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c312a6f0 0x55f3c312cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcf93fff640 10 client.25726 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.041+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c312a6f0 0x55f3c312cae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5637bee872cf87be server_cookie=c01f84aaaa02ee58 in_seq=0 out_seq=0 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcf93fff640 1 -- 172.21.15.37:0/859195113 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3c3082390 con 0x55f3c312a6f0 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcf93fff640 10 client.25726 handle_command_reply: tid=0 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 client.25726 shutdown 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 msgr2=0x55f3c311a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c311a160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009720 tx=0x7fcfa0009260 comp rx=0 tx=0).stop 2024-04-23T18:25:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 shutdown_connections 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c312a6f0 0x55f3c312cae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c31179b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310ff40 0x55f3c3117450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/859195113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c311a160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcf93fff640 0 client.25726 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 >> 172.21.15.37:0/859195113 conn(0x55f3c310bb30 msgr2=0x55f3c31118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 shutdown_connections 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/859195113 wait complete. 2024-04-23T18:25:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.042+0000 7fcfbdb7f740 20 client.25726 trim_cache size 0 max 16384 2024-04-23T18:25:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 Processor -- start 2024-04-23T18:25:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 -- start start 2024-04-23T18:25:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c31262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c3119c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c31320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b2820 con 0x55f3c310ff40 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b0090 con 0x55f3c311a800 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.057+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41adee0 con 0x55f3c310f620 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c3119c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c3119c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47200/0 (socket says 172.21.15.37:47200) 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 -- 172.21.15.37:0/1631095435 learned_addr learned my addr 172.21.15.37:0/1631095435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c31262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c31320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 -- 172.21.15.37:0/1631095435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 msgr2=0x55f3c31262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c31262f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 -- 172.21.15.37:0/1631095435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 msgr2=0x55f3c31320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c31320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 -- 172.21.15.37:0/1631095435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c310fd70 con 0x55f3c310ff40 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c31320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c3119c90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcfa802d020 tx=0x7fcfa80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8680a7edb72168a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcf9e7fc640 1 -- 172.21.15.37:0/1631095435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8037040 con 0x55f3c310ff40 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcf9e7fc640 1 -- 172.21.15.37:0/1631095435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8030900 con 0x55f3c310ff40 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.058+0000 7fcf9e7fc640 1 -- 172.21.15.37:0/1631095435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8003df0 con 0x55f3c310ff40 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1631095435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 msgr2=0x55f3c3119c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c3119c90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcfa802d020 tx=0x7fcfa80058d0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1631095435 shutdown_connections 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c311a800 0x55f3c31320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c310ff40 0x55f3c3119c90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/1631095435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310f620 0x55f3c31262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1631095435 >> 172.21.15.37:0/1631095435 conn(0x55f3c310bb30 msgr2=0x55f3c31129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1631095435 shutdown_connections 2024-04-23T18:25:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/1631095435 wait complete. 2024-04-23T18:25:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.059+0000 7fcfbdb7f740 1 -- start start 2024-04-23T18:25:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 Processor -- start 2024-04-23T18:25:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310f620 0x55f3c3188c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310ff40 0x55f3c31891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c311dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41ae550 con 0x55f3c311a800 2024-04-23T18:25:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41af9c0 con 0x55f3c310f620 2024-04-23T18:25:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41aec20 con 0x55f3c310ff40 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfbdb7f740 10 client.0 _ll_register_callbacks cb 0x7fcfa001d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c311dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310ff40 0x55f3c31891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310f620 0x55f3c3188c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb8c07640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310ff40 0x55f3c31891d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:41670/0 (socket says 172.21.15.37:41670) 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c311dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47210/0 (socket says 172.21.15.37:47210) 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb8c07640 1 -- 172.21.15.37:0/4162720011 learned_addr learned my addr 172.21.15.37:0/4162720011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 -- 172.21.15.37:0/4162720011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310ff40 msgr2=0x55f3c31891d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310ff40 0x55f3c31891d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 -- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310f620 msgr2=0x55f3c3188c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310f620 0x55f3c3188c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.060+0000 7fcfb3fff640 1 -- 172.21.15.37:0/4162720011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c311e2c0 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfb3fff640 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c311dbd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0001eb0 tx=0x7fcfa0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30ef30308b02070 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcf93fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcf93fff640 1 -- 172.21.15.37:0/4162720011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa003a040 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcf93fff640 1 -- 172.21.15.37:0/4162720011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fcfa001da30 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa8030900 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcf93fff640 1 -- 172.21.15.37:0/4162720011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcfa0022040 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 10 client.16203 fetch_fsmap learned FSMap version 32 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3c3190440 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcf93fff640 1 -- 172.21.15.37:0/4162720011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8030900 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcf93fff640 1 -- 172.21.15.37:0/4162720011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcfa0040040 con 0x55f3c311a800 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 10 client.16203 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:25:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 20 client.16203 populate_metadata read hostname 'smithi037' 2024-04-23T18:25:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c3190a50 0x55f3c3192e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 4 client.16203 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:25:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.061+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3c307c200 con 0x55f3c3190a50 2024-04-23T18:25:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.062+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c3190a50 0x55f3c3192e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.062+0000 7fcf93fff640 10 client.16203 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.062+0000 7fcfb8c07640 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c3190a50 0x55f3c3192e70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4572379e857b7d26 server_cookie=c1cdb397c85fc793 in_seq=0 out_seq=0 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.083+0000 7fcf93fff640 1 -- 172.21.15.37:0/4162720011 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3c307c200 con 0x55f3c3190a50 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.083+0000 7fcf93fff640 10 client.16203 handle_command_reply: tid=0 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 client.16203 shutdown 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 msgr2=0x55f3c311dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c311dbd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0001eb0 tx=0x7fcfa0009260 comp rx=0 tx=0).stop 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 shutdown_connections 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f3c3190a50 0x55f3c3192e70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f3c311a800 0x55f3c311dbd0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f3c310ff40 0x55f3c31891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcf93fff640 0 client.16203 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:25:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 --2- 172.21.15.37:0/4162720011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c310f620 0x55f3c3188c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 >> 172.21.15.37:0/4162720011 conn(0x55f3c310bb30 msgr2=0x55f3c31252e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 shutdown_connections 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 1 -- 172.21.15.37:0/4162720011 wait complete. 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.084+0000 7fcfbdb7f740 20 client.16203 trim_cache size 0 max 16384 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:25:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcfa40a3e40 msgr2=0x7fcfa40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcfa40a3e40 0x7fcfa40a6300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ce580 tx=0x7fcfac0057a0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 msgr2=0x7fcfb419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 0x7fcfb419a7c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00eb40 tx=0x7fcfac00ef40 comp rx=0 tx=0).stop 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 shutdown_connections 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcfa40a3e40 0x7fcfa40a6300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfb410e930 0x7fcfb419a7c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcfb41016a0 0x7fcfb41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 --2- 172.21.15.37:0/784185476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb4100d80 0x7fcfb41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 >> 172.21.15.37:0/784185476 conn(0x7fcfb4077fd0 msgr2=0x7fcfb410fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 shutdown_connections 2024-04-23T18:25:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:25:57.085+0000 7fcfbae6b640 1 -- 172.21.15.37:0/784185476 wait complete. 2024-04-23T18:25:57.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 342177 4.53 MB/sec execute 122 sec latency 22.508 ms 2024-04-23T18:25:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:57 smithi172 ceph-mon[32456]: pgmap v169: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 737 KiB/s wr, 81 op/s 2024-04-23T18:25:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:57 smithi173 ceph-mon[32906]: pgmap v169: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 737 KiB/s wr, 81 op/s 2024-04-23T18:25:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:57 smithi037 ceph-mon[29404]: pgmap v169: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 737 KiB/s wr, 81 op/s 2024-04-23T18:25:58.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 344447 4.61 MB/sec execute 123 sec latency 15.422 ms 2024-04-23T18:25:59.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 346268 4.61 MB/sec execute 124 sec latency 17.378 ms 2024-04-23T18:26:00.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:25:59 smithi172 ceph-mon[32456]: pgmap v170: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 682 KiB/s wr, 56 op/s 2024-04-23T18:26:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:25:59 smithi173 ceph-mon[32906]: pgmap v170: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 682 KiB/s wr, 56 op/s 2024-04-23T18:26:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:25:59 smithi037 ceph-mon[29404]: pgmap v170: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 682 KiB/s wr, 56 op/s 2024-04-23T18:26:00.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 348148 4.68 MB/sec execute 125 sec latency 16.543 ms 2024-04-23T18:26:01.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 349141 4.69 MB/sec execute 126 sec latency 26.497 ms 2024-04-23T18:26:02.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:01 smithi172 ceph-mon[32456]: pgmap v171: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 682 KiB/s wr, 56 op/s 2024-04-23T18:26:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:01 smithi173 ceph-mon[32906]: pgmap v171: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 682 KiB/s wr, 56 op/s 2024-04-23T18:26:02.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:01 smithi037 ceph-mon[29404]: pgmap v171: 97 pgs: 97 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 682 KiB/s wr, 56 op/s 2024-04-23T18:26:02.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 351426 4.76 MB/sec execute 127 sec latency 50.242 ms 2024-04-23T18:26:03.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 352681 4.76 MB/sec execute 128 sec latency 24.192 ms 2024-04-23T18:26:04.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:03 smithi172 ceph-mon[32456]: pgmap v172: 97 pgs: 97 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-04-23T18:26:04.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:03 smithi173 ceph-mon[32906]: pgmap v172: 97 pgs: 97 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-04-23T18:26:04.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:03 smithi037 ceph-mon[29404]: pgmap v172: 97 pgs: 97 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-04-23T18:26:04.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355004 4.83 MB/sec execute 129 sec latency 12.946 ms 2024-04-23T18:26:05.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 356561 4.84 MB/sec execute 130 sec latency 14.000 ms 2024-04-23T18:26:06.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:06 smithi172 ceph-mon[32456]: pgmap v173: 97 pgs: 97 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-23T18:26:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:06 smithi173 ceph-mon[32906]: pgmap v173: 97 pgs: 97 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-23T18:26:06.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:06 smithi037 ceph-mon[29404]: pgmap v173: 97 pgs: 97 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-23T18:26:06.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 358664 4.91 MB/sec execute 131 sec latency 14.589 ms 2024-04-23T18:26:07.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 360651 4.92 MB/sec execute 132 sec latency 20.768 ms 2024-04-23T18:26:08.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:08 smithi172 ceph-mon[32456]: pgmap v174: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 290 op/s 2024-04-23T18:26:08.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:08 smithi173 ceph-mon[32906]: pgmap v174: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 290 op/s 2024-04-23T18:26:08.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:08 smithi037 ceph-mon[29404]: pgmap v174: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 290 op/s 2024-04-23T18:26:08.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 362323 4.98 MB/sec execute 133 sec latency 12.897 ms 2024-04-23T18:26:09.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 364240 4.99 MB/sec execute 134 sec latency 18.045 ms 2024-04-23T18:26:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 -- 172.21.15.37:0/4011685117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8100700 msgr2=0x7fb3a8100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/4011685117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8100700 0x7fb3a8100ae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb398009900 tx=0x7fb39802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 -- 172.21.15.37:0/4011685117 shutdown_connections 2024-04-23T18:26:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/4011685117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a8107310 0x7fb3a810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/4011685117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81010b0 0x7fb3a8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/4011685117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8100700 0x7fb3a8100ae0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 -- 172.21.15.37:0/4011685117 >> 172.21.15.37:0/4011685117 conn(0x7fb3a8077390 msgr2=0x7fb3a80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 -- 172.21.15.37:0/4011685117 shutdown_connections 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.176+0000 7fb3acc9a640 1 -- 172.21.15.37:0/4011685117 wait complete. 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 Processor -- start 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 -- start start 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a8100700 0x7fb3a819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 0x7fb3a819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81985b0 0x7fb3a8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8112250 con 0x7fb3a8107310 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81120d0 con 0x7fb3a81985b0 2024-04-23T18:26:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3acc9a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81123d0 con 0x7fb3a8100700 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 0x7fb3a819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 0x7fb3a819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57456/0 (socket says 172.21.15.37:57456) 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb39ffff640 1 -- 172.21.15.37:0/2333385552 learned_addr learned my addr 172.21.15.37:0/2333385552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3a6d9d640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81985b0 0x7fb3a8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.177+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a8100700 0x7fb3a819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3a6d9d640 1 -- 172.21.15.37:0/2333385552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a8100700 msgr2=0x7fb3a819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3a6d9d640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a8100700 0x7fb3a819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3a6d9d640 1 -- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 msgr2=0x7fb3a819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3a6d9d640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 0x7fb3a819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3a6d9d640 1 -- 172.21.15.37:0/2333385552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a81990d0 con 0x7fb3a81985b0 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 0x7fb3a819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3a6d9d640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81985b0 0x7fb3a8198a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e940 tx=0x7fb3a000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb39f7fe640 1 -- 172.21.15.37:0/2333385552 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0004d20 con 0x7fb3a81985b0 2024-04-23T18:26:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a81993c0 con 0x7fb3a81985b0 2024-04-23T18:26:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb39f7fe640 1 -- 172.21.15.37:0/2333385552 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a0005020 con 0x7fb3a81985b0 2024-04-23T18:26:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb39f7fe640 1 -- 172.21.15.37:0/2333385552 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00116d0 con 0x7fb3a81985b0 2024-04-23T18:26:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.178+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a81a5610 con 0x7fb3a81985b0 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.180+0000 7fb39f7fe640 1 -- 172.21.15.37:0/2333385552 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3a0005460 con 0x7fb3a81985b0 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 Processor -- start 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 -- start start 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 0x56393c2a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2b2800 0x56393c2b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56393c20c970 con 0x56393c2a7620 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56393c1a94a0 con 0x56393c2a7f40 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56393c208930 con 0x56393c2b2800 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb39f7fe640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb3940a3dc0 0x7fb3940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 0x56393c2a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 0x56393c2a7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57468/0 (socket says 172.21.15.37:57468) 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb39ffff640 1 -- 172.21.15.37:0/3331292343 learned_addr learned my addr 172.21.15.37:0/3331292343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3a659c640 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb39ffff640 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2b2800 0x56393c2b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.181+0000 7fb3a6d9d640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb3940a3dc0 0x7fb3940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb39f7fe640 1 -- 172.21.15.37:0/2333385552 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fb3a00d0890 con 0x7fb3a81985b0 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a659c640 1 -- 172.21.15.37:0/3331292343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2b2800 msgr2=0x56393c2b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a659c640 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2b2800 0x56393c2b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a659c640 1 -- 172.21.15.37:0/3331292343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 msgr2=0x56393c2a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a659c640 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 0x56393c2a7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a659c640 1 -- 172.21.15.37:0/3331292343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56393bf1f210 con 0x56393c2a7f40 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb39ffff640 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 0x56393c2a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a6d9d640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb3940a3dc0 0x7fb3940a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3a009c570 tx=0x7fb3a001c040 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb3a659c640 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b22c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb39802aa00 tx=0x7fb398004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d389fe68c32e468 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb39cff9640 1 -- 172.21.15.37:0/3331292343 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398035040 con 0x56393c2a7f40 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb39cff9640 1 -- 172.21.15.37:0/3331292343 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3980054f0 con 0x56393c2a7f40 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.182+0000 7fb39cff9640 1 -- 172.21.15.37:0/3331292343 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398004e40 con 0x56393c2a7f40 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 -- 172.21.15.37:0/3331292343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 msgr2=0x56393c2b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b22c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb39802aa00 tx=0x7fb398004190 comp rx=0 tx=0).stop 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 -- 172.21.15.37:0/3331292343 shutdown_connections 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2b2800 0x56393c2b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b22c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/3331292343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7620 0x56393c2a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 -- 172.21.15.37:0/3331292343 >> 172.21.15.37:0/3331292343 conn(0x56393c2a3b30 msgr2=0x56393c2a3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 -- 172.21.15.37:0/3331292343 shutdown_connections 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 -- 172.21.15.37:0/3331292343 wait complete. 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 -- start start 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.183+0000 7fb3af9b4740 1 Processor -- start 2024-04-23T18:26:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56393c2b53d0 con 0x56393c2a7f40 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56393c1a70b0 con 0x56393c2b2800 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56393c2b5550 con 0x56393c2a7620 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3af9b4740 10 client.0 _ll_register_callbacks cb 0x56393c194d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57478/0 (socket says 172.21.15.37:57478) 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3a659c640 1 -- 172.21.15.37:0/2293827494 learned_addr learned my addr 172.21.15.37:0/2293827494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 -- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 msgr2=0x56393c2af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 -- 172.21.15.37:0/2293827494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 msgr2=0x56393c2af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 -- 172.21.15.37:0/2293827494 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56393bfeb620 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2b2130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x56393c2a4f50 tx=0x7fb390009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4499d8368e1d6ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb38dffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.184+0000 7fb38dffb640 1 -- 172.21.15.37:0/2293827494 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900035f0 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb38dffb640 1 -- 172.21.15.37:0/2293827494 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fb390003790 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3980054f0 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb38dffb640 1 -- 172.21.15.37:0/2293827494 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb390003990 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 10 client.25744 fetch_fsmap learned FSMap version 32 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56393c2c12e0 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb38dffb640 1 -- 172.21.15.37:0/2293827494 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3980054f0 con 0x56393c2a7620 2024-04-23T18:26:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb38dffb640 1 -- 172.21.15.37:0/2293827494 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb3900047a0 con 0x56393c2a7620 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 10 client.25744 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 20 client.25744 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c2c27f0 0x56393c2c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 4 client.25744 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56393c21a390 con 0x56393c2c27f0 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.185+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c2c27f0 0x56393c2c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.186+0000 7fb38dffb640 10 client.25744 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.186+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c2c27f0 0x56393c2c4c60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc99452d72ecf08 server_cookie=635a4627ea002c12 in_seq=0 out_seq=0 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb38dffb640 1 -- 172.21.15.37:0/2293827494 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56393c21a390 con 0x56393c2c27f0 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb38dffb640 10 client.25744 handle_command_reply: tid=0 2024-04-23T18:26:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 client.25744 shutdown 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 msgr2=0x56393c2b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2b2130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x56393c2a4f50 tx=0x7fb390009d40 comp rx=0 tx=0).stop 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 shutdown_connections 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c2c27f0 0x56393c2c4c60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb38dffb640 0 client.25744 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2293827494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2b2130 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 >> 172.21.15.37:0/2293827494 conn(0x56393c2a3b30 msgr2=0x56393c2a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 shutdown_connections 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2293827494 wait complete. 2024-04-23T18:26:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.187+0000 7fb3af9b4740 20 client.25744 trim_cache size 0 max 16384 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 Processor -- start 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 -- start start 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2be290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81b0530 con 0x56393c2a7f40 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81adda0 con 0x56393c2b2800 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81abbf0 con 0x56393c2a7620 2024-04-23T18:26:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57488/0 (socket says 172.21.15.37:57488) 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 -- 172.21.15.37:0/2951626926 learned_addr learned my addr 172.21.15.37:0/2951626926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2be290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 -- 172.21.15.37:0/2951626926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 msgr2=0x56393c2be290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2be290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 -- 172.21.15.37:0/2951626926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 msgr2=0x56393c2b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 -- 172.21.15.37:0/2951626926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56393c2afcc0 con 0x56393c2a7f40 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.202+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2aa140 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb398004160 tx=0x7fb3980040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badc27122ba48fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb38ffff640 1 -- 172.21.15.37:0/2951626926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398035040 con 0x56393c2a7f40 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb38ffff640 1 -- 172.21.15.37:0/2951626926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb398058070 con 0x56393c2a7f40 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb38ffff640 1 -- 172.21.15.37:0/2951626926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980035f0 con 0x56393c2a7f40 2024-04-23T18:26:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2951626926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 msgr2=0x56393c2aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2aa140 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb398004160 tx=0x7fb3980040f0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2951626926 shutdown_connections 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2b2800 0x56393c2b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2a7f40 0x56393c2aa140 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2951626926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2be290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2951626926 >> 172.21.15.37:0/2951626926 conn(0x56393c2a3b30 msgr2=0x56393c2a9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2951626926 shutdown_connections 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.203+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2951626926 wait complete. 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 -- start start 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 Processor -- start 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 0x56393c2b6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81ac260 con 0x56393c2b2800 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81ad6d0 con 0x56393c2a7f40 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81ac930 con 0x56393c2a7620 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3af9b4740 10 client.0 _ll_register_callbacks cb 0x7fb390013280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 0x56393c2b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 0x56393c2b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57500/0 (socket says 172.21.15.37:57500) 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 -- 172.21.15.37:0/2427790914 learned_addr learned my addr 172.21.15.37:0/2427790914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 -- 172.21.15.37:0/2427790914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 msgr2=0x56393c2bab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2bab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 -- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 msgr2=0x56393c2b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.204+0000 7fb39ffff640 1 -- 172.21.15.37:0/2427790914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56393c2b6b90 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb39ffff640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 0x56393c2b6360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb390001eb0 tx=0x7fb390009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9808595038f7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb38dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb38dffb640 1 -- 172.21.15.37:0/2427790914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390003b30 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb38dffb640 1 -- 172.21.15.37:0/2427790914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fb390003cd0 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb398058070 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb38dffb640 1 -- 172.21.15.37:0/2427790914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb390004de0 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 10 client.16221 fetch_fsmap learned FSMap version 32 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56393c328d20 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb38dffb640 1 -- 172.21.15.37:0/2427790914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb398058070 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb38dffb640 1 -- 172.21.15.37:0/2427790914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb39003d740 con 0x56393c2b2800 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 10 client.16221 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 20 client.16221 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c329410 0x56393c32b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 4 client.16221 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56393c214200 con 0x56393c329410 2024-04-23T18:26:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.205+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c329410 0x56393c32b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.206+0000 7fb3a659c640 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c329410 0x56393c32b830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c9598653062d0e server_cookie=51c1f77163b1f5c5 in_seq=0 out_seq=0 2024-04-23T18:26:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.206+0000 7fb38dffb640 10 client.16221 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb38dffb640 1 -- 172.21.15.37:0/2427790914 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56393c214200 con 0x56393c329410 2024-04-23T18:26:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb38dffb640 10 client.16221 handle_command_reply: tid=0 2024-04-23T18:26:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 client.16221 shutdown 2024-04-23T18:26:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 msgr2=0x56393c2b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 0x56393c2b6360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb390001eb0 tx=0x7fb390009d40 comp rx=0 tx=0).stop 2024-04-23T18:26:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 shutdown_connections 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56393c329410 0x56393c32b830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393c2b2800 0x56393c2b6360 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56393c2a7f40 0x56393c2b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 --2- 172.21.15.37:0/2427790914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56393c2a7620 0x56393c2bab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 >> 172.21.15.37:0/2427790914 conn(0x56393c2a3b30 msgr2=0x56393c2a2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb38dffb640 0 client.16221 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 shutdown_connections 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 1 -- 172.21.15.37:0/2427790914 wait complete. 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.238+0000 7fb3af9b4740 20 client.16221 trim_cache size 0 max 16384 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:26:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:26:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:26:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:26:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb3940a3dc0 msgr2=0x7fb3940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb3940a3dc0 0x7fb3940a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3a009c570 tx=0x7fb3a001c040 comp rx=0 tx=0).stop 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81985b0 msgr2=0x7fb3a8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81985b0 0x7fb3a8198a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e940 tx=0x7fb3a000ee00 comp rx=0 tx=0).stop 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 shutdown_connections 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb3940a3dc0 0x7fb3940a6280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a81985b0 0x7fb3a8198a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a8107310 0x7fb3a819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 --2- 172.21.15.37:0/2333385552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a8100700 0x7fb3a819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 >> 172.21.15.37:0/2333385552 conn(0x7fb3a8077390 msgr2=0x7fb3a80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 shutdown_connections 2024-04-23T18:26:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:10.239+0000 7fb3acc9a640 1 -- 172.21.15.37:0/2333385552 wait complete. 2024-04-23T18:26:10.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:10 smithi172 ceph-mon[32456]: pgmap v175: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-23T18:26:10.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:10 smithi173 ceph-mon[32906]: pgmap v175: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-23T18:26:10.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:10 smithi037 ceph-mon[29404]: pgmap v175: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-23T18:26:10.822 INFO:tasks.workunit.client.0.smithi037.stdout: 1 365873 5.05 MB/sec execute 135 sec latency 13.224 ms 2024-04-23T18:26:11.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 367898 5.07 MB/sec execute 136 sec latency 19.295 ms 2024-04-23T18:26:12.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:12 smithi172 ceph-mon[32456]: pgmap v176: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-23T18:26:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:12 smithi173 ceph-mon[32906]: pgmap v176: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-23T18:26:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:12 smithi037 ceph-mon[29404]: pgmap v176: 97 pgs: 97 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2024-04-23T18:26:12.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 369432 5.11 MB/sec execute 137 sec latency 19.019 ms 2024-04-23T18:26:12.941 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:26:12.941 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-04-23T18:26:12.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 -- 172.21.15.37:0/2396018234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164101a50 msgr2=0x7f4164105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 --2- 172.21.15.37:0/2396018234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164101a50 0x7f4164105a90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4154009900 tx=0x7f415402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 -- 172.21.15.37:0/2396018234 shutdown_connections 2024-04-23T18:26:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 --2- 172.21.15.37:0/2396018234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4164106170 0x7f416410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 --2- 172.21.15.37:0/2396018234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164101a50 0x7f4164105a90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 --2- 172.21.15.37:0/2396018234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41641010a0 0x7f4164101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 -- 172.21.15.37:0/2396018234 >> 172.21.15.37:0/2396018234 conn(0x7f4164077f50 msgr2=0x7f4164078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 -- 172.21.15.37:0/2396018234 shutdown_connections 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.058+0000 7f416c75d640 1 -- 172.21.15.37:0/2396018234 wait complete. 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 Processor -- start 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 -- start start 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41641010a0 0x7f416419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 0x7f416419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41641985e0 0x7f4164198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4164111090 con 0x7f4164106170 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4164110f10 con 0x7f41641985e0 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416c75d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4164111210 con 0x7f41641010a0 2024-04-23T18:26:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f4169cf8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 0x7f416419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f4169cf8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 0x7f416419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57514/0 (socket says 172.21.15.37:57514) 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f4169cf8640 1 -- 172.21.15.37:0/259332278 learned_addr learned my addr 172.21.15.37:0/259332278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416a4f9640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41641010a0 0x7f416419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f416acfa640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41641985e0 0x7f4164198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.059+0000 7f4169cf8640 1 -- 172.21.15.37:0/259332278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41641010a0 msgr2=0x7f416419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f4169cf8640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41641010a0 0x7f416419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f4169cf8640 1 -- 172.21.15.37:0/259332278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41641985e0 msgr2=0x7f4164198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f4169cf8640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41641985e0 0x7f4164198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f4169cf8640 1 -- 172.21.15.37:0/259332278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4164199100 con 0x7f4164106170 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f416a4f9640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41641010a0 0x7f416419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f4169cf8640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 0x7f416419e920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4154032fd0 tx=0x7f4154031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415403f030 con 0x7f4164106170 2024-04-23T18:26:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4164199390 con 0x7f4164106170 2024-04-23T18:26:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41540043d0 con 0x7f4164106170 2024-04-23T18:26:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154035070 con 0x7f4164106170 2024-04-23T18:26:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.060+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41641a5590 con 0x7f4164106170 2024-04-23T18:26:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.061+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4164068d70 con 0x7f4164106170 2024-04-23T18:26:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.062+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4154042030 con 0x7f4164106170 2024-04-23T18:26:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.063+0000 7f415b7fe640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41500a3d20 0x7f41500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.063+0000 7f416a4f9640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41500a3d20 0x7f41500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.063+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f41540f0e60 con 0x7f4164106170 2024-04-23T18:26:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.063+0000 7f416a4f9640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41500a3d20 0x7f41500a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4160009fc0 tx=0x7f4160005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.067+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41540bd4f0 con 0x7f4164106170 2024-04-23T18:26:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.303+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4164101a50 con 0x7f4164106170 2024-04-23T18:26:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.304+0000 7f415b7fe640 1 -- 172.21.15.37:0/259332278 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11881 (secure 0 0 0) 0x7f41540bcdf0 con 0x7f4164106170 2024-04-23T18:26:13.297 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:26:13.297 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:24:43.883083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:26:13.297 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 32 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.305+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41500a3d20 msgr2=0x7f41500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.305+0000 7f416c75d640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41500a3d20 0x7f41500a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4160009fc0 tx=0x7f4160005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.305+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 msgr2=0x7f416419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.305+0000 7f416c75d640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 0x7f416419e920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4154032fd0 tx=0x7f4154031a20 comp rx=0 tx=0).stop 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 shutdown_connections 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f41500a3d20 0x7f41500a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41641985e0 0x7f4164198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4164106170 0x7f416419e920 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 --2- 172.21.15.37:0/259332278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41641010a0 0x7f416419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 >> 172.21.15.37:0/259332278 conn(0x7f4164077f50 msgr2=0x7f4164104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 shutdown_connections 2024-04-23T18:26:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.306+0000 7f416c75d640 1 -- 172.21.15.37:0/259332278 wait complete. 2024-04-23T18:26:13.309 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 84.6 --await-for 42.0' 2024-04-23T18:26:13.309 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 84.6 --await-for 42.0 2024-04-23T18:26:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 -- 172.21.15.37:0/1023625376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d8101a50 msgr2=0x7fa1d8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 --2- 172.21.15.37:0/1023625376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d8101a50 0x7fa1d8105a90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009900 tx=0x7fa1c802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 -- 172.21.15.37:0/1023625376 shutdown_connections 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 --2- 172.21.15.37:0/1023625376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d8106170 0x7fa1d810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 --2- 172.21.15.37:0/1023625376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d8101a50 0x7fa1d8105a90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 --2- 172.21.15.37:0/1023625376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d81010a0 0x7fa1d8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 -- 172.21.15.37:0/1023625376 >> 172.21.15.37:0/1023625376 conn(0x7fa1d8077f50 msgr2=0x7fa1d8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.432+0000 7fa1dde10640 1 -- 172.21.15.37:0/1023625376 shutdown_connections 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.433+0000 7fa1dde10640 1 -- 172.21.15.37:0/1023625376 wait complete. 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.433+0000 7fa1dde10640 1 Processor -- start 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.433+0000 7fa1dde10640 1 -- start start 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.433+0000 7fa1dde10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 0x7fa1d81976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.433+0000 7fa1dde10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d8106170 0x7fa1d8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1dde10640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d819bf60 0x7fa1d819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1dde10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81110b0 con 0x7fa1d81010a0 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1dde10640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8110f30 con 0x7fa1d8106170 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1dde10640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8111230 con 0x7fa1d819bf60 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 0x7fa1d81976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 0x7fa1d81976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57524/0 (socket says 172.21.15.37:57524) 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 -- 172.21.15.37:0/3662344817 learned_addr learned my addr 172.21.15.37:0/3662344817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 -- 172.21.15.37:0/3662344817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d819bf60 msgr2=0x7fa1d819c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d819bf60 0x7fa1d819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 -- 172.21.15.37:0/3662344817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d8106170 msgr2=0x7fa1d8197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d8106170 0x7fa1d8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 -- 172.21.15.37:0/3662344817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d819ca50 con 0x7fa1d81010a0 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 0x7fa1d81976a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9f0 tx=0x7fa1c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d4ff9640 1 -- 172.21.15.37:0/3662344817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000cd80 con 0x7fa1d81010a0 2024-04-23T18:26:13.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d4ff9640 1 -- 172.21.15.37:0/3662344817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c00108e0 con 0x7fa1d81010a0 2024-04-23T18:26:13.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d819cd40 con 0x7fa1d81010a0 2024-04-23T18:26:13.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1d4ff9640 1 -- 172.21.15.37:0/3662344817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0018980 con 0x7fa1d81010a0 2024-04-23T18:26:13.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.434+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d81a4580 con 0x7fa1d81010a0 2024-04-23T18:26:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.437+0000 7fa1d4ff9640 1 -- 172.21.15.37:0/3662344817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa1c0010430 con 0x7fa1d81010a0 2024-04-23T18:26:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.437+0000 7fa1e0c02740 1 Processor -- start 2024-04-23T18:26:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.437+0000 7fa1e0c02740 1 -- start start 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556fe431f370 con 0x556fe43ba690 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556fe42bc2e0 con 0x556fe43b9d70 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556fe431b330 con 0x556fe43c72a0 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43ba150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:43690/0 (socket says 172.21.15.37:43690) 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d7fff640 1 -- 172.21.15.37:0/571896101 learned_addr learned my addr 172.21.15.37:0/571896101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d4ff9640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1c40a3da0 0x7fa1c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 -- 172.21.15.37:0/571896101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 msgr2=0x556fe43c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 -- 172.21.15.37:0/571896101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 msgr2=0x556fe43ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 -- 172.21.15.37:0/571896101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe40fdd80 con 0x556fe43ba690 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1c40a3da0 0x7fa1c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d4ff9640 1 -- 172.21.15.37:0/3662344817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa1c00d0c00 con 0x7fa1d81010a0 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.438+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43c6d60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00ce340 tx=0x7fa1c0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809ebd2068ac5ab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1cdffb640 1 -- 172.21.15.37:0/571896101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c009c6d0 con 0x556fe43ba690 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1cdffb640 1 -- 172.21.15.37:0/571896101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c0010720 con 0x556fe43ba690 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1cdffb640 1 -- 172.21.15.37:0/571896101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c009c4f0 con 0x556fe43ba690 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 -- 172.21.15.37:0/571896101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 msgr2=0x556fe43c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1c40a3da0 0x7fa1c40a6260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00049d0 tx=0x7fa1d0004a40 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43c6d60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00ce340 tx=0x7fa1c0024000 comp rx=0 tx=0).stop 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 -- 172.21.15.37:0/571896101 shutdown_connections 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43c6d60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/571896101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 -- 172.21.15.37:0/571896101 >> 172.21.15.37:0/571896101 conn(0x556fe43b63c0 msgr2=0x556fe43b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 -- 172.21.15.37:0/571896101 shutdown_connections 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.439+0000 7fa1e0c02740 1 -- 172.21.15.37:0/571896101 wait complete. 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 -- start start 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 Processor -- start 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 0x556fe43c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556fe43ca000 con 0x556fe43b9d70 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556fe42b9fb0 con 0x556fe43ba690 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556fe43ca180 con 0x556fe43c72a0 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1e0c02740 10 client.0 _ll_register_callbacks cb 0x556fe42b99a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 0x556fe43c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 0x556fe43c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57536/0 (socket says 172.21.15.37:57536) 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1d77fe640 1 -- 172.21.15.37:0/1938282110 learned_addr learned my addr 172.21.15.37:0/1938282110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.440+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1d77fe640 1 -- 172.21.15.37:0/1938282110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 msgr2=0x556fe43c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1d77fe640 1 -- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 msgr2=0x556fe43c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1d77fe640 1 -- 172.21.15.37:0/1938282110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe415e510 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 0x556fe43c4250 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9c0 tx=0x7fa1c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6129b5d66e52d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/1938282110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c009c4f0 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/1938282110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa1c00d43e0 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556fe432cd90 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/1938282110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1c009c6d0 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1e0c02740 10 client.16239 fetch_fsmap learned FSMap version 32 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556fe3ff1670 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.441+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/1938282110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556fe432cd90 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/1938282110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa1c00d8040 con 0x556fe43b9d70 2024-04-23T18:26:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1e0c02740 10 client.16239 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1e0c02740 10 client.16239 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1e0c02740 20 client.16239 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe43d7480 0x556fe43d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1e0c02740 4 client.16239 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe4326c00 con 0x556fe43d7480 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe43d7480 0x556fe43d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe43d7480 0x556fe43d9870 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe93a67f2517fd1 server_cookie=1103a634c91b7d6d in_seq=0 out_seq=0 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.442+0000 7fa1b6ffd640 10 client.16239 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/1938282110 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fe4326c00 con 0x556fe43d7480 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1b6ffd640 10 client.16239 handle_command_reply: tid=0 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 client.16239 shutdown 2024-04-23T18:26:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 msgr2=0x556fe43c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 0x556fe43c4250 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9c0 tx=0x7fa1c0000bc0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 shutdown_connections 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe43d7480 0x556fe43d9870 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1b6ffd640 0 client.16239 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/1938282110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43b9d70 0x556fe43c4250 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.443+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 >> 172.21.15.37:0/1938282110 conn(0x556fe43b63c0 msgr2=0x556fe43bc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.444+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 shutdown_connections 2024-04-23T18:26:13.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.444+0000 7fa1e0c02740 1 -- 172.21.15.37:0/1938282110 wait complete. 2024-04-23T18:26:13.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.444+0000 7fa1e0c02740 20 client.16239 trim_cache size 0 max 16384 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 Processor -- start 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 -- start start 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81ae3b0 con 0x556fe43ba690 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81abc20 con 0x556fe43b9d70 2024-04-23T18:26:13.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81a9a70 con 0x556fe43c72a0 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57546/0 (socket says 172.21.15.37:57546) 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 -- 172.21.15.37:0/3694118962 learned_addr learned my addr 172.21.15.37:0/3694118962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 -- 172.21.15.37:0/3694118962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 msgr2=0x556fe43bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43bc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 -- 172.21.15.37:0/3694118962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 msgr2=0x556fe43d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 -- 172.21.15.37:0/3694118962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe43c3280 con 0x556fe43ba690 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.470+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43b5d80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009900 tx=0x7fa1c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820a9e38860af6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/3694118962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8035040 con 0x556fe43ba690 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/3694118962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8005040 con 0x556fe43ba690 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/3694118962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8004900 con 0x556fe43ba690 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 -- 172.21.15.37:0/3694118962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 msgr2=0x556fe43b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43b5d80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009900 tx=0x7fa1c8000bc0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 -- 172.21.15.37:0/3694118962 shutdown_connections 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43c72a0 0x556fe43bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43ba690 0x556fe43b5d80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/3694118962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43b9d70 0x556fe43d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 -- 172.21.15.37:0/3694118962 >> 172.21.15.37:0/3694118962 conn(0x556fe43b63c0 msgr2=0x556fe43bb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.471+0000 7fa1e0c02740 1 -- 172.21.15.37:0/3694118962 shutdown_connections 2024-04-23T18:26:13.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 -- 172.21.15.37:0/3694118962 wait complete. 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 -- start start 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 Processor -- start 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43b9d70 0x556fe43cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 0x556fe43cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81aa0e0 con 0x556fe43c72a0 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81ab550 con 0x556fe43ba690 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81aa7b0 con 0x556fe43b9d70 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1e0c02740 10 client.0 _ll_register_callbacks cb 0x7fa1c00d4190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:13.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 0x556fe43cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 0x556fe43cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57554/0 (socket says 172.21.15.37:57554) 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1d7fff640 1 -- 172.21.15.37:0/519638951 learned_addr learned my addr 172.21.15.37:0/519638951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43b9d70 0x556fe43cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.472+0000 7fa1d6ffd640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1d7fff640 1 -- 172.21.15.37:0/519638951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43b9d70 msgr2=0x556fe43cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43b9d70 0x556fe43cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1d7fff640 1 -- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 msgr2=0x556fe43cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1d7fff640 1 -- 172.21.15.37:0/519638951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe43cb940 con 0x556fe43c72a0 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1d7fff640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 0x556fe43cb110 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0012990 tx=0x7fa1d0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ab7fb67a0363f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/519638951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0020d50 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/519638951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa1d0029440 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c8005040 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/519638951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1d0029640 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1e0c02740 10 client.16245 fetch_fsmap learned FSMap version 32 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556fe443d810 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.473+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/519638951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8005040 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/519638951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa1d0027020 con 0x556fe43c72a0 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1e0c02740 10 client.16245 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1e0c02740 10 client.16245 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:26:13.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1e0c02740 20 client.16245 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:13.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe443e090 0x556fe44404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1e0c02740 4 client.16245 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.6, "await_for": 42.0}] 2024-04-23T18:26:13.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.6, "await_for": 42.0}) -- 0x556fe42a2ca0 con 0x556fe443e090 2024-04-23T18:26:13.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe443e090 0x556fe44404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1d77fe640 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe443e090 0x556fe44404b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b009ceb1e79d9737 server_cookie=8cdd7c2e8e940cfa in_seq=0 out_seq=0 2024-04-23T18:26:13.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.474+0000 7fa1b6ffd640 10 client.16245 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1b6ffd640 1 -- 172.21.15.37:0/519638951 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x556fe42a2ca0 con 0x556fe443e090 2024-04-23T18:26:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1b6ffd640 10 client.16245 handle_command_reply: tid=0 2024-04-23T18:26:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 client.16245 shutdown 2024-04-23T18:26:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 msgr2=0x556fe43cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 0x556fe43cb110 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0012990 tx=0x7fa1d0009500 comp rx=0 tx=0).stop 2024-04-23T18:26:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 shutdown_connections 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556fe443e090 0x556fe44404b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe43c72a0 0x556fe43cb110 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fe43ba690 0x556fe43cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 --2- 172.21.15.37:0/519638951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556fe43b9d70 0x556fe43cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1b6ffd640 0 client.16245 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 >> 172.21.15.37:0/519638951 conn(0x556fe43b63c0 msgr2=0x556fe43bc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 shutdown_connections 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.554+0000 7fa1e0c02740 1 -- 172.21.15.37:0/519638951 wait complete. 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1e0c02740 20 client.16245 trim_cache size 0 max 16384 2024-04-23T18:26:13.549 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 32, 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 22, 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "cca99928": { 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 22, 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:26:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.6, 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:13.552 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:26:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1c40a3da0 msgr2=0x7fa1c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1c40a3da0 0x7fa1c40a6260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00049d0 tx=0x7fa1d0004a40 comp rx=0 tx=0).stop 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 msgr2=0x7fa1d81976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 0x7fa1d81976a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9f0 tx=0x7fa1c000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 shutdown_connections 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1c40a3da0 0x7fa1c40a6260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d819bf60 0x7fa1d819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d8106170 0x7fa1d8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 --2- 172.21.15.37:0/3662344817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d81010a0 0x7fa1d81976a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 >> 172.21.15.37:0/3662344817 conn(0x7fa1d8077f50 msgr2=0x7fa1d80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 shutdown_connections 2024-04-23T18:26:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:13.555+0000 7fa1dde10640 1 -- 172.21.15.37:0/3662344817 wait complete. 2024-04-23T18:26:13.559 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cca99928', quiesce took 0.6 seconds. Will release after: 24.6 2024-04-23T18:26:13.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 5.11 MB/sec execute 138 sec latency 256.626 ms 2024-04-23T18:26:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:14 smithi172 ceph-mon[32456]: pgmap v177: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-04-23T18:26:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:14 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/259332278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:14 smithi173 ceph-mon[32906]: pgmap v177: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-04-23T18:26:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:14 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/259332278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:14.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:14 smithi037 ceph-mon[29404]: pgmap v177: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-04-23T18:26:14.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:14 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/259332278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:14.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 5.08 MB/sec execute 139 sec latency 1256.751 ms 2024-04-23T18:26:15.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 5.04 MB/sec execute 140 sec latency 2256.840 ms 2024-04-23T18:26:16.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:16 smithi172 ceph-mon[32456]: pgmap v178: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-04-23T18:26:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:16 smithi173 ceph-mon[32906]: pgmap v178: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-04-23T18:26:16.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:16 smithi037 ceph-mon[29404]: pgmap v178: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 212 op/s 2024-04-23T18:26:16.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 5.00 MB/sec execute 141 sec latency 3256.913 ms 2024-04-23T18:26:17.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.97 MB/sec execute 142 sec latency 4257.037 ms 2024-04-23T18:26:18.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:18 smithi172 ceph-mon[32456]: pgmap v179: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-04-23T18:26:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:18 smithi173 ceph-mon[32906]: pgmap v179: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-04-23T18:26:18.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:18 smithi037 ceph-mon[29404]: pgmap v179: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-04-23T18:26:18.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.93 MB/sec execute 143 sec latency 5257.102 ms 2024-04-23T18:26:19.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:19 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:26:19.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:19 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:26:19.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:19 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:26:19.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.90 MB/sec execute 144 sec latency 6257.207 ms 2024-04-23T18:26:20.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:20 smithi172 ceph-mon[32456]: pgmap v180: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-23T18:26:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:20 smithi173 ceph-mon[32906]: pgmap v180: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-23T18:26:20.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:20 smithi037 ceph-mon[29404]: pgmap v180: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-23T18:26:20.823 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.87 MB/sec execute 145 sec latency 7257.347 ms 2024-04-23T18:26:21.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.83 MB/sec execute 146 sec latency 8257.467 ms 2024-04-23T18:26:22.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:22 smithi172 ceph-mon[32456]: pgmap v181: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-23T18:26:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:22 smithi173 ceph-mon[32906]: pgmap v181: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-23T18:26:22.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:22 smithi037 ceph-mon[29404]: pgmap v181: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-04-23T18:26:22.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.80 MB/sec execute 147 sec latency 9257.572 ms 2024-04-23T18:26:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 -- 172.21.15.37:0/2318224668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6281061b0 msgr2=0x7fa62810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 --2- 172.21.15.37:0/2318224668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6281061b0 0x7fa62810dd80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa620009ca0 tx=0x7fa62002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 -- 172.21.15.37:0/2318224668 shutdown_connections 2024-04-23T18:26:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 --2- 172.21.15.37:0/2318224668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6281061b0 0x7fa62810dd80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 --2- 172.21.15.37:0/2318224668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa628101a90 0x7fa628105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 --2- 172.21.15.37:0/2318224668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6281010e0 0x7fa6281014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 -- 172.21.15.37:0/2318224668 >> 172.21.15.37:0/2318224668 conn(0x7fa6280771d0 msgr2=0x7fa6280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 -- 172.21.15.37:0/2318224668 shutdown_connections 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.338+0000 7fa62f35d640 1 -- 172.21.15.37:0/2318224668 wait complete. 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 Processor -- start 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 -- start start 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6281010e0 0x7fa628199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6281061b0 0x7fa628199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 0x7fa6281a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa628111120 con 0x7fa628199f80 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa628110fa0 con 0x7fa6281010e0 2024-04-23T18:26:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62f35d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281112a0 con 0x7fa6281061b0 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62d8fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 0x7fa6281a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62d8fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 0x7fa6281a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54144/0 (socket says 172.21.15.37:54144) 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.339+0000 7fa62d8fa640 1 -- 172.21.15.37:0/1826508079 learned_addr learned my addr 172.21.15.37:0/1826508079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6281010e0 0x7fa628199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6281061b0 0x7fa628199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d8fa640 1 -- 172.21.15.37:0/1826508079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6281061b0 msgr2=0x7fa628199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d8fa640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6281061b0 0x7fa628199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d8fa640 1 -- 172.21.15.37:0/1826508079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6281010e0 msgr2=0x7fa628199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d8fa640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6281010e0 0x7fa628199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d8fa640 1 -- 172.21.15.37:0/1826508079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6281a4d00 con 0x7fa628199f80 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6281010e0 0x7fa628199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62d8fa640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 0x7fa6281a4580 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa620005cc0 tx=0x7fa62002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa61e7fc640 1 -- 172.21.15.37:0/1826508079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620003bf0 con 0x7fa628199f80 2024-04-23T18:26:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa61e7fc640 1 -- 172.21.15.37:0/1826508079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa620003d90 con 0x7fa628199f80 2024-04-23T18:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa61e7fc640 1 -- 172.21.15.37:0/1826508079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62002fbc0 con 0x7fa628199f80 2024-04-23T18:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6281a4f90 con 0x7fa628199f80 2024-04-23T18:26:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.340+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a54b0 con 0x7fa628199f80 2024-04-23T18:26:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.343+0000 7fa61e7fc640 1 -- 172.21.15.37:0/1826508079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa62002fd60 con 0x7fa628199f80 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.343+0000 7fa632076740 1 Processor -- start 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.343+0000 7fa632076740 1 -- start start 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa632076740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa632076740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e1b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa632076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa632076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563552d75970 con 0x563552e1b800 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa632076740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563552d124a0 con 0x563552e10f40 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa632076740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563552d71930 con 0x563552e10620 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54152/0 (socket says 172.21.15.37:54152) 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 -- 172.21.15.37:0/839961457 learned_addr learned my addr 172.21.15.37:0/839961457 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e1b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa61e7fc640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa6140a3dc0 0x7fa6140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 -- 172.21.15.37:0/839961457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 msgr2=0x563552e10a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62d8fa640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa6140a3dc0 0x7fa6140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e10a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 -- 172.21.15.37:0/839961457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 msgr2=0x563552e1b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e1b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa62c8f8640 1 -- 172.21.15.37:0/839961457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563552a88210 con 0x563552e1b800 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.344+0000 7fa61e7fc640 1 -- 172.21.15.37:0/1826508079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa6200f1f00 con 0x7fa628199f80 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.345+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1dbf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa618013780 tx=0x7fa61800e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647994ff42653e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.345+0000 7fa6137fe640 1 -- 172.21.15.37:0/839961457 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61800b660 con 0x563552e1b800 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.345+0000 7fa6137fe640 1 -- 172.21.15.37:0/839961457 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61800bcb0 con 0x563552e1b800 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.345+0000 7fa6137fe640 1 -- 172.21.15.37:0/839961457 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61801ecc0 con 0x563552e1b800 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.345+0000 7fa62d8fa640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa6140a3dc0 0x7fa6140a6280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa6200bdc40 tx=0x7fa6200b33d0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 -- 172.21.15.37:0/839961457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 msgr2=0x563552e1dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1dbf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa618013780 tx=0x7fa61800e330 comp rx=0 tx=0).stop 2024-04-23T18:26:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 -- 172.21.15.37:0/839961457 shutdown_connections 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1dbf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e1b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 --2- 172.21.15.37:0/839961457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e10a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 -- 172.21.15.37:0/839961457 >> 172.21.15.37:0/839961457 conn(0x563552e0cb30 msgr2=0x563552e0cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 -- 172.21.15.37:0/839961457 shutdown_connections 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 -- 172.21.15.37:0/839961457 wait complete. 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 -- start start 2024-04-23T18:26:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.346+0000 7fa632076740 1 Processor -- start 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e1b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e18940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563552e1e430 con 0x563552e1b800 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563552d100b0 con 0x563552e10f40 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563552e1e5b0 con 0x563552e10620 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa632076740 10 client.0 _ll_register_callbacks cb 0x563552cfdd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e18940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e18940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54164/0 (socket says 172.21.15.37:54164) 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3896099304 learned_addr learned my addr 172.21.15.37:0/3896099304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e1b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3896099304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 msgr2=0x563552e1b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e1b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 msgr2=0x563552e18400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e18400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3896099304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563552b54620 con 0x563552e1b800 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e18940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa618005ed0 tx=0x7fa618001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e8aa3144599378 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.347+0000 7fa610ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa610ff9640 1 -- 172.21.15.37:0/3896099304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618022020 con 0x563552e1b800 2024-04-23T18:26:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa610ff9640 1 -- 172.21.15.37:0/3896099304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa618013070 con 0x563552e1b800 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61800bcb0 con 0x563552e1b800 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa610ff9640 1 -- 172.21.15.37:0/3896099304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa618013270 con 0x563552e1b800 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 10 client.16257 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563552e2a350 con 0x563552e1b800 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa610ff9640 1 -- 172.21.15.37:0/3896099304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61800bcb0 con 0x563552e1b800 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa610ff9640 1 -- 172.21.15.37:0/3896099304 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fa61800f040 con 0x563552e1b800 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 10 client.16257 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 20 client.16257 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e2b750 0x563552e2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 4 client.16257 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563552d83390 con 0x563552e2b750 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.348+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e2b750 0x563552e2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.349+0000 7fa610ff9640 10 client.16257 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.349+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e2b750 0x563552e2db40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385f2fd600ff12fb server_cookie=695e1e651ec42c85 in_seq=0 out_seq=0 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa610ff9640 1 -- 172.21.15.37:0/3896099304 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563552d83390 con 0x563552e2b750 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa610ff9640 10 client.16257 handle_command_reply: tid=0 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 client.16257 shutdown 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 msgr2=0x563552e18940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e18940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa618005ed0 tx=0x7fa618001e50 comp rx=0 tx=0).stop 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 shutdown_connections 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e2b750 0x563552e2db40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e18940 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e18400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa610ff9640 0 client.16257 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 --2- 172.21.15.37:0/3896099304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e1b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 >> 172.21.15.37:0/3896099304 conn(0x563552e0cb30 msgr2=0x563552e12860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 shutdown_connections 2024-04-23T18:26:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 1 -- 172.21.15.37:0/3896099304 wait complete. 2024-04-23T18:26:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.350+0000 7fa632076740 20 client.16257 trim_cache size 0 max 16384 2024-04-23T18:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 Processor -- start 2024-04-23T18:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 -- start start 2024-04-23T18:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e0c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281b0470 con 0x563552e1b800 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281adce0 con 0x563552e10f40 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.365+0000 7fa632076740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281abb30 con 0x563552e10620 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54180/0 (socket says 172.21.15.37:54180) 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 -- 172.21.15.37:0/1842373753 learned_addr learned my addr 172.21.15.37:0/1842373753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e0c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 -- 172.21.15.37:0/1842373753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 msgr2=0x563552e0c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e0c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 -- 172.21.15.37:0/1842373753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 msgr2=0x563552e126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 -- 172.21.15.37:0/1842373753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563552e119e0 con 0x563552e1b800 2024-04-23T18:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1ac40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa618013570 tx=0x7fa618015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f350975a92d865e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa612ffd640 1 -- 172.21.15.37:0/1842373753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61803d070 con 0x563552e1b800 2024-04-23T18:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa612ffd640 1 -- 172.21.15.37:0/1842373753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61800e7d0 con 0x563552e1b800 2024-04-23T18:26:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.366+0000 7fa612ffd640 1 -- 172.21.15.37:0/1842373753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618022040 con 0x563552e1b800 2024-04-23T18:26:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 -- 172.21.15.37:0/1842373753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 msgr2=0x563552e1ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1ac40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa618013570 tx=0x7fa618015040 comp rx=0 tx=0).stop 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 -- 172.21.15.37:0/1842373753 shutdown_connections 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e1ac40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 --2- 172.21.15.37:0/1842373753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e0c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 -- 172.21.15.37:0/1842373753 >> 172.21.15.37:0/1842373753 conn(0x563552e0cb30 msgr2=0x563552e11fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 -- 172.21.15.37:0/1842373753 shutdown_connections 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 -- 172.21.15.37:0/1842373753 wait complete. 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.367+0000 7fa632076740 1 -- start start 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 Processor -- start 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e23d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281ac1a0 con 0x563552e1b800 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281ad610 con 0x563552e10f40 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281ac870 con 0x563552e10620 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa632076740 10 client.0 _ll_register_callbacks cb 0x7fa618013680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e14670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54190/0 (socket says 172.21.15.37:54190) 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3662760213 learned_addr learned my addr 172.21.15.37:0/3662760213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e23d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3662760213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 msgr2=0x563552e23800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e23800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 msgr2=0x563552e23d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e23d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62c8f8640 1 -- 172.21.15.37:0/3662760213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563552e14bb0 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.368+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e23d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa62c8f8640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e14670 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa6180057c0 tx=0x7fa61801e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5330696fd21f3b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa610ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa610ff9640 1 -- 172.21.15.37:0/3662760213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61804a060 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa610ff9640 1 -- 172.21.15.37:0/3662760213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa61800bb60 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61800e7d0 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa610ff9640 1 -- 172.21.15.37:0/3662760213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa618022070 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 10 client.16263 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563552e912d0 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa610ff9640 1 -- 172.21.15.37:0/3662760213 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61800e7d0 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa610ff9640 1 -- 172.21.15.37:0/3662760213 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fa618043070 con 0x563552e1b800 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 10 client.16263 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 20 client.16263 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e919c0 0x563552e93de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 4 client.16263 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.369+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563552e18cd0 con 0x563552e919c0 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.370+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e919c0 0x563552e93de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.370+0000 7fa62d0f9640 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e919c0 0x563552e93de0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1d4952c8fe707f server_cookie=5179da4ae7eed4da in_seq=0 out_seq=0 2024-04-23T18:26:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.370+0000 7fa610ff9640 10 client.16263 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa610ff9640 1 -- 172.21.15.37:0/3662760213 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563552e18cd0 con 0x563552e919c0 2024-04-23T18:26:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa610ff9640 10 client.16263 handle_command_reply: tid=0 2024-04-23T18:26:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 client.16263 shutdown 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 msgr2=0x563552e14670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e14670 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa6180057c0 tx=0x7fa61801e770 comp rx=0 tx=0).stop 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 shutdown_connections 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563552e919c0 0x563552e93de0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563552e1b800 0x563552e14670 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563552e10f40 0x563552e23d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 --2- 172.21.15.37:0/3662760213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563552e10620 0x563552e23800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa610ff9640 0 client.16263 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.388+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 >> 172.21.15.37:0/3662760213 conn(0x563552e0cb30 msgr2=0x563552e11f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 shutdown_connections 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa632076740 1 -- 172.21.15.37:0/3662760213 wait complete. 2024-04-23T18:26:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa632076740 20 client.16263 trim_cache size 0 max 16384 2024-04-23T18:26:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:26:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:26:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:26:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa6140a3dc0 msgr2=0x7fa6140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa6140a3dc0 0x7fa6140a6280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa6200bdc40 tx=0x7fa6200b33d0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 msgr2=0x7fa6281a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 0x7fa6281a4580 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa620005cc0 tx=0x7fa62002f380 comp rx=0 tx=0).stop 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 shutdown_connections 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa6140a3dc0 0x7fa6140a6280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa628199f80 0x7fa6281a4580 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6281061b0 0x7fa628199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 --2- 172.21.15.37:0/1826508079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6281010e0 0x7fa628199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 >> 172.21.15.37:0/1826508079 conn(0x7fa6280771d0 msgr2=0x7fa6280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 shutdown_connections 2024-04-23T18:26:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:23.389+0000 7fa62f35d640 1 -- 172.21.15.37:0/1826508079 wait complete. 2024-04-23T18:26:23.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.77 MB/sec execute 148 sec latency 10257.677 ms 2024-04-23T18:26:24.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:24 smithi172 ceph-mon[32456]: pgmap v182: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-04-23T18:26:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:24 smithi173 ceph-mon[32906]: pgmap v182: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-04-23T18:26:24.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:24 smithi037 ceph-mon[29404]: pgmap v182: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-04-23T18:26:24.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.73 MB/sec execute 149 sec latency 11257.788 ms 2024-04-23T18:26:25.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.70 MB/sec execute 150 sec latency 12257.892 ms 2024-04-23T18:26:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:26 smithi172 ceph-mon[32456]: pgmap v183: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 731 KiB/s wr, 58 op/s 2024-04-23T18:26:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:26 smithi173 ceph-mon[32906]: pgmap v183: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 731 KiB/s wr, 58 op/s 2024-04-23T18:26:26.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:26 smithi037 ceph-mon[29404]: pgmap v183: 97 pgs: 97 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 731 KiB/s wr, 58 op/s 2024-04-23T18:26:26.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.67 MB/sec execute 151 sec latency 13258.008 ms 2024-04-23T18:26:27.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.64 MB/sec execute 152 sec latency 14258.142 ms 2024-04-23T18:26:28.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:28 smithi172 ceph-mon[32456]: pgmap v184: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 737 KiB/s wr, 67 op/s 2024-04-23T18:26:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:28 smithi173 ceph-mon[32906]: pgmap v184: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 737 KiB/s wr, 67 op/s 2024-04-23T18:26:28.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:28 smithi037 ceph-mon[29404]: pgmap v184: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 737 KiB/s wr, 67 op/s 2024-04-23T18:26:28.824 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.61 MB/sec execute 153 sec latency 15258.248 ms 2024-04-23T18:26:29.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.58 MB/sec execute 154 sec latency 16258.358 ms 2024-04-23T18:26:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:30 smithi172 ceph-mon[32456]: pgmap v185: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-23T18:26:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:30 smithi173 ceph-mon[32906]: pgmap v185: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-23T18:26:30.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:30 smithi037 ceph-mon[29404]: pgmap v185: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-23T18:26:30.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.55 MB/sec execute 155 sec latency 17258.475 ms 2024-04-23T18:26:31.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.52 MB/sec execute 156 sec latency 18258.599 ms 2024-04-23T18:26:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:32 smithi172 ceph-mon[32456]: pgmap v186: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-23T18:26:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:32 smithi173 ceph-mon[32906]: pgmap v186: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-23T18:26:32.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:32 smithi037 ceph-mon[29404]: pgmap v186: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-23T18:26:32.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.49 MB/sec execute 157 sec latency 19258.694 ms 2024-04-23T18:26:33.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.46 MB/sec execute 158 sec latency 20258.796 ms 2024-04-23T18:26:34.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:34 smithi172 ceph-mon[32456]: pgmap v187: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 603 KiB/s wr, 37 op/s 2024-04-23T18:26:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:34 smithi173 ceph-mon[32906]: pgmap v187: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 603 KiB/s wr, 37 op/s 2024-04-23T18:26:34.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:34 smithi037 ceph-mon[29404]: pgmap v187: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 603 KiB/s wr, 37 op/s 2024-04-23T18:26:34.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.44 MB/sec execute 159 sec latency 21258.896 ms 2024-04-23T18:26:35.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.41 MB/sec execute 160 sec latency 22259.038 ms 2024-04-23T18:26:36.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:36 smithi172 ceph-mon[32456]: pgmap v188: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 578 KiB/s wr, 29 op/s 2024-04-23T18:26:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:36 smithi173 ceph-mon[32906]: pgmap v188: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 578 KiB/s wr, 29 op/s 2024-04-23T18:26:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:36 smithi037 ceph-mon[29404]: pgmap v188: 97 pgs: 97 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 578 KiB/s wr, 29 op/s 2024-04-23T18:26:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 -- 172.21.15.37:0/3282547010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d0103b20 msgr2=0x7fa1d010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 --2- 172.21.15.37:0/3282547010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d0103b20 0x7fa1d010c000 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009900 tx=0x7fa1c002a480 comp rx=0 tx=0).stop 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 -- 172.21.15.37:0/3282547010 shutdown_connections 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 --2- 172.21.15.37:0/3282547010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d010c5d0 0x7fa1d010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 --2- 172.21.15.37:0/3282547010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d0103b20 0x7fa1d010c000 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 --2- 172.21.15.37:0/3282547010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d0103200 0x7fa1d01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 -- 172.21.15.37:0/3282547010 >> 172.21.15.37:0/3282547010 conn(0x7fa1d00771d0 msgr2=0x7fa1d00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 -- 172.21.15.37:0/3282547010 shutdown_connections 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.487+0000 7fa1d669a640 1 -- 172.21.15.37:0/3282547010 wait complete. 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.488+0000 7fa1d669a640 1 Processor -- start 2024-04-23T18:26:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.488+0000 7fa1d669a640 1 -- start start 2024-04-23T18:26:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d669a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d0103200 0x7fa1d0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d669a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d0103b20 0x7fa1d0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d669a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 0x7fa1d019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d669a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01121f0 con 0x7fa1d010c5d0 2024-04-23T18:26:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d669a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0112070 con 0x7fa1d0103200 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d669a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0112370 con 0x7fa1d0103b20 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 0x7fa1d019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 0x7fa1d019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37170/0 (socket says 172.21.15.37:37170) 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 -- 172.21.15.37:0/2338548860 learned_addr learned my addr 172.21.15.37:0/2338548860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1cffff640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d0103200 0x7fa1d0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d0103b20 0x7fa1d0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 -- 172.21.15.37:0/2338548860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d0103b20 msgr2=0x7fa1d0198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d0103b20 0x7fa1d0198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 -- 172.21.15.37:0/2338548860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d0103200 msgr2=0x7fa1d0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d0103200 0x7fa1d0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 -- 172.21.15.37:0/2338548860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d019d640 con 0x7fa1d010c5d0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1d4c37640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 0x7fa1d019cf10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800dac0 tx=0x7fa1c800dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.489+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/2338548860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8017070 con 0x7fa1d010c5d0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.490+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/2338548860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8003e20 con 0x7fa1d010c5d0 2024-04-23T18:26:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.490+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d019d930 con 0x7fa1d010c5d0 2024-04-23T18:26:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.490+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/2338548860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8010460 con 0x7fa1d010c5d0 2024-04-23T18:26:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.490+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a51c0 con 0x7fa1d010c5d0 2024-04-23T18:26:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.492+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/2338548860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa1c8004020 con 0x7fa1d010c5d0 2024-04-23T18:26:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.492+0000 7fa1d93b5740 1 Processor -- start 2024-04-23T18:26:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.492+0000 7fa1d93b5740 1 -- start start 2024-04-23T18:26:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.492+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d18e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.492+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d236d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.492+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 0x560035d26000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560035c79970 con 0x560035d23c10 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560035c164a0 con 0x560035d18a30 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560035c75930 con 0x560035d19350 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cd7fa640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1b80a3e40 0x7fa1b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 0x560035d26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 0x560035d26000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37182/0 (socket says 172.21.15.37:37182) 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/274311814 learned_addr learned my addr 172.21.15.37:0/274311814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d18e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1d4c37640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1b80a3e40 0x7fa1b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cffff640 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cd7fa640 1 -- 172.21.15.37:0/2338548860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa1c80d03a0 con 0x7fa1d010c5d0 2024-04-23T18:26:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/274311814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 msgr2=0x560035d236d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d236d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/274311814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 msgr2=0x560035d18e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d18e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/274311814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56003598c210 con 0x560035d23c10 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 0x560035d26000 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa1c002a9a0 tx=0x7fa1c0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ea6ec6c089aed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.493+0000 7fa1d4c37640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1b80a3e40 0x7fa1b80a6300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800da90 tx=0x7fa1c80043c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1b67fc640 1 -- 172.21.15.37:0/274311814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0035070 con 0x560035d23c10 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1b67fc640 1 -- 172.21.15.37:0/274311814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c00035f0 con 0x560035d23c10 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1b67fc640 1 -- 172.21.15.37:0/274311814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00044e0 con 0x560035d23c10 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 -- 172.21.15.37:0/274311814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 msgr2=0x560035d26000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 0x560035d26000 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa1c002a9a0 tx=0x7fa1c0005db0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 -- 172.21.15.37:0/274311814 shutdown_connections 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d23c10 0x560035d26000 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d236d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/274311814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d18e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 -- 172.21.15.37:0/274311814 >> 172.21.15.37:0/274311814 conn(0x560035d14f40 msgr2=0x560035d15320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 -- 172.21.15.37:0/274311814 shutdown_connections 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.494+0000 7fa1d93b5740 1 -- 172.21.15.37:0/274311814 wait complete. 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 -- start start 2024-04-23T18:26:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 Processor -- start 2024-04-23T18:26:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d18a30 0x560035d23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d20860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d20dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560035d267e0 con 0x560035d19350 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560035c140b0 con 0x560035d23c10 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560035d26960 con 0x560035d18a30 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1d93b5740 10 client.0 _ll_register_callbacks cb 0x560035c01d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d20860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d20860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37198/0 (socket says 172.21.15.37:37198) 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.495+0000 7fa1cffff640 1 -- 172.21.15.37:0/1715680799 learned_addr learned my addr 172.21.15.37:0/1715680799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d20dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d18a30 0x560035d23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cffff640 1 -- 172.21.15.37:0/1715680799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d18a30 msgr2=0x560035d23500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cffff640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d18a30 0x560035d23500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cffff640 1 -- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 msgr2=0x560035d20dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cffff640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d20dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cffff640 1 -- 172.21.15.37:0/1715680799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560035a58620 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d18a30 0x560035d23500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1cffff640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d20860 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4010a50 tx=0x7fa1c4010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfb56b8280a9cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1af7fe640 1 -- 172.21.15.37:0/1715680799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c401e060 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c00035f0 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1af7fe640 1 -- 172.21.15.37:0/1715680799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa1c400fbd0 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1af7fe640 1 -- 172.21.15.37:0/1715680799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1c400fdd0 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1d93b5740 10 client.16275 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560035d32830 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.496+0000 7fa1af7fe640 1 -- 172.21.15.37:0/1715680799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c00035f0 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1af7fe640 1 -- 172.21.15.37:0/1715680799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fa1c400e4d0 con 0x560035d19350 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1d93b5740 10 client.16275 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1d93b5740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1d93b5740 20 client.16275 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d33d60 0x560035d36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1d93b5740 4 client.16275 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560035c87390 con 0x560035d33d60 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.497+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d33d60 0x560035d36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.498+0000 7fa1af7fe640 10 client.16275 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.498+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d33d60 0x560035d36150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7050841bac8aa8 server_cookie=9991153621cd6e52 in_seq=0 out_seq=0 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.498+0000 7fa1af7fe640 1 -- 172.21.15.37:0/1715680799 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560035c87390 con 0x560035d33d60 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.498+0000 7fa1af7fe640 10 client.16275 handle_command_reply: tid=0 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.498+0000 7fa1d93b5740 1 client.16275 shutdown 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 msgr2=0x560035d20860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d20860 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4010a50 tx=0x7fa1c4010f10 comp rx=0 tx=0).stop 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 shutdown_connections 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d33d60 0x560035d36150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d20dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d20860 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1af7fe640 0 client.16275 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/1715680799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d18a30 0x560035d23500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 >> 172.21.15.37:0/1715680799 conn(0x560035d14f40 msgr2=0x560035d0fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 shutdown_connections 2024-04-23T18:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 1 -- 172.21.15.37:0/1715680799 wait complete. 2024-04-23T18:26:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.499+0000 7fa1d93b5740 20 client.16275 trim_cache size 0 max 16384 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 Processor -- start 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 -- start start 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d0fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d23c10 0x560035d3bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01ae150 con 0x560035d19350 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01ab9c0 con 0x560035d18a30 2024-04-23T18:26:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01a9830 con 0x560035d23c10 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d3bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37208/0 (socket says 172.21.15.37:37208) 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 -- 172.21.15.37:0/3485344028 learned_addr learned my addr 172.21.15.37:0/3485344028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d0fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 -- 172.21.15.37:0/3485344028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d23c10 msgr2=0x560035d3bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d23c10 0x560035d3bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 -- 172.21.15.37:0/3485344028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 msgr2=0x560035d0fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d0fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 -- 172.21.15.37:0/3485344028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560035d0c2e0 con 0x560035d19350 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.514+0000 7fa1cffff640 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d3bab0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c400a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c0b87ae81c9c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1b5ffb640 1 -- 172.21.15.37:0/3485344028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400fbd0 con 0x560035d19350 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1b5ffb640 1 -- 172.21.15.37:0/3485344028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c400eb60 con 0x560035d19350 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1b5ffb640 1 -- 172.21.15.37:0/3485344028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4004660 con 0x560035d19350 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 -- 172.21.15.37:0/3485344028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 msgr2=0x560035d3bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d3bab0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c400a8d0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 -- 172.21.15.37:0/3485344028 shutdown_connections 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d23c10 0x560035d3bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d19350 0x560035d3bab0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/3485344028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d18a30 0x560035d0fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 -- 172.21.15.37:0/3485344028 >> 172.21.15.37:0/3485344028 conn(0x560035d14f40 msgr2=0x560035d0f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 -- 172.21.15.37:0/3485344028 shutdown_connections 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.515+0000 7fa1d93b5740 1 -- 172.21.15.37:0/3485344028 wait complete. 2024-04-23T18:26:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 -- start start 2024-04-23T18:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 Processor -- start 2024-04-23T18:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 0x560035d2bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d2c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01a9e80 con 0x560035d18a30 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01ab2f0 con 0x560035d23c10 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01aa550 con 0x560035d19350 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1d93b5740 10 client.0 _ll_register_callbacks cb 0x7fa1c4010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 0x560035d2bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 0x560035d2bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37218/0 (socket says 172.21.15.37:37218) 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/2609307793 learned_addr learned my addr 172.21.15.37:0/2609307793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cffff640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d2c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/2609307793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 msgr2=0x560035d2c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d2c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 msgr2=0x560035d291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.516+0000 7fa1cf7fe640 1 -- 172.21.15.37:0/2609307793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560035d21130 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1cffff640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d2c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1cf7fe640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 0x560035d2bc30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0031440 tx=0x7fa1c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e06984b0dfa6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1af7fe640 1 -- 172.21.15.37:0/2609307793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0058020 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1af7fe640 1 -- 172.21.15.37:0/2609307793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa1c0004000 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c400eb60 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1af7fe640 1 -- 172.21.15.37:0/2609307793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa1c0035040 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 10 client.16281 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560035d99d70 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1af7fe640 1 -- 172.21.15.37:0/2609307793 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c400eb60 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1af7fe640 1 -- 172.21.15.37:0/2609307793 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fa1c00521d0 con 0x560035d18a30 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 10 client.16281 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 20 client.16281 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d9a100 0x560035d9c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 4 client.16281 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.517+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560035c81200 con 0x560035d9a100 2024-04-23T18:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.518+0000 7fa1cffff640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d9a100 0x560035d9c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.518+0000 7fa1af7fe640 10 client.16281 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.518+0000 7fa1cffff640 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d9a100 0x560035d9c520 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d079bf06900dd1a server_cookie=86bd22ddbb85c247 in_seq=0 out_seq=0 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1af7fe640 1 -- 172.21.15.37:0/2609307793 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560035c81200 con 0x560035d9a100 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1af7fe640 10 client.16281 handle_command_reply: tid=0 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 client.16281 shutdown 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 msgr2=0x560035d2bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 0x560035d2bc30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0031440 tx=0x7fa1c0005e80 comp rx=0 tx=0).stop 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 shutdown_connections 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x560035d9a100 0x560035d9c520 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560035d23c10 0x560035d291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560035d19350 0x560035d2c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 --2- 172.21.15.37:0/2609307793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560035d18a30 0x560035d2bc30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1af7fe640 0 client.16281 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 >> 172.21.15.37:0/2609307793 conn(0x560035d14f40 msgr2=0x560035d0f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 shutdown_connections 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 1 -- 172.21.15.37:0/2609307793 wait complete. 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.531+0000 7fa1d93b5740 20 client.16281 trim_cache size 0 max 16384 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:26:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:26:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1b80a3e40 msgr2=0x7fa1b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1b80a3e40 0x7fa1b80a6300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800da90 tx=0x7fa1c80043c0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 msgr2=0x7fa1d019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 0x7fa1d019cf10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800dac0 tx=0x7fa1c800dec0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 shutdown_connections 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa1b80a3e40 0x7fa1b80a6300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d010c5d0 0x7fa1d019cf10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1d0103b20 0x7fa1d0198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 --2- 172.21.15.37:0/2338548860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1d0103200 0x7fa1d0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 >> 172.21.15.37:0/2338548860 conn(0x7fa1d00771d0 msgr2=0x7fa1d00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 shutdown_connections 2024-04-23T18:26:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:36.532+0000 7fa1d669a640 1 -- 172.21.15.37:0/2338548860 wait complete. 2024-04-23T18:26:36.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.38 MB/sec execute 161 sec latency 23259.148 ms 2024-04-23T18:26:37.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370477 4.35 MB/sec execute 162 sec latency 24259.202 ms 2024-04-23T18:26:38.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:38.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 -- 172.21.15.37:0/4080325287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c10c570 msgr2=0x7fe55c10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 --2- 172.21.15.37:0/4080325287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c10c570 0x7fe55c10e9e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe554009ca0 tx=0x7fe55402a950 comp rx=0 tx=0).stop 2024-04-23T18:26:38.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 -- 172.21.15.37:0/4080325287 shutdown_connections 2024-04-23T18:26:38.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 --2- 172.21.15.37:0/4080325287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c10c570 0x7fe55c10e9e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 --2- 172.21.15.37:0/4080325287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c103b50 0x7fe55c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 --2- 172.21.15.37:0/4080325287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103230 0x7fe55c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 -- 172.21.15.37:0/4080325287 >> 172.21.15.37:0/4080325287 conn(0x7fe55c077fb0 msgr2=0x7fe55c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 -- 172.21.15.37:0/4080325287 shutdown_connections 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.281+0000 7fe561508640 1 -- 172.21.15.37:0/4080325287 wait complete. 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 Processor -- start 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 -- start start 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 0x7fe55c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103b50 0x7fe55c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c10c570 0x7fe55c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1121d0 con 0x7fe55c103230 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c112050 con 0x7fe55c103b50 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe561508640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c112350 con 0x7fe55c10c570 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe55affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 0x7fe55c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe55affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 0x7fe55c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37228/0 (socket says 172.21.15.37:37228) 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.282+0000 7fe55affd640 1 -- 172.21.15.37:0/362142737 learned_addr learned my addr 172.21.15.37:0/362142737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe553fff640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103b50 0x7fe55c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55b7fe640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c10c570 0x7fe55c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55affd640 1 -- 172.21.15.37:0/362142737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c10c570 msgr2=0x7fe55c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55affd640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c10c570 0x7fe55c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55affd640 1 -- 172.21.15.37:0/362142737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103b50 msgr2=0x7fe55c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55affd640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103b50 0x7fe55c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55affd640 1 -- 172.21.15.37:0/362142737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c19d6f0 con 0x7fe55c103230 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe553fff640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103b50 0x7fe55c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55b7fe640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c10c570 0x7fe55c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe55affd640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 0x7fe55c198750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00d950 tx=0x7fe54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c017070 con 0x7fe55c103230 2024-04-23T18:26:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe54c009c30 con 0x7fe55c103230 2024-04-23T18:26:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19d9e0 con 0x7fe55c103230 2024-04-23T18:26:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00f620 con 0x7fe55c103230 2024-04-23T18:26:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.283+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1a5270 con 0x7fe55c103230 2024-04-23T18:26:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.284+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c0691c0 con 0x7fe55c103230 2024-04-23T18:26:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.285+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe54c010460 con 0x7fe55c103230 2024-04-23T18:26:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.286+0000 7fe558ff9640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe5480a3da0 0x7fe5480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.286+0000 7fe553fff640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe5480a3da0 0x7fe5480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.286+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe54c0d0500 con 0x7fe55c103230 2024-04-23T18:26:38.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.286+0000 7fe553fff640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe5480a3da0 0x7fe5480a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe55c100900 tx=0x7fe544005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.291+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe54c09cb30 con 0x7fe55c103230 2024-04-23T18:26:38.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:38 smithi172 ceph-mon[32456]: pgmap v189: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-04-23T18:26:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:38 smithi173 ceph-mon[32906]: pgmap v189: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-04-23T18:26:38.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:38 smithi037 ceph-mon[29404]: pgmap v189: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-04-23T18:26:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.517+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe55c199a50 con 0x7fe55c103230 2024-04-23T18:26:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.518+0000 7fe558ff9640 1 -- 172.21.15.37:0/362142737 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11879 (secure 0 0 0) 0x7fe54c009de0 con 0x7fe55c103230 2024-04-23T18:26:38.511 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:26:38.512 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:26:17.565566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24484,24509,24479]},"id":1}]} 2024-04-23T18:26:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-23T18:26:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.519+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe5480a3da0 msgr2=0x7fe5480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.519+0000 7fe561508640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe5480a3da0 0x7fe5480a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe55c100900 tx=0x7fe544005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.519+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 msgr2=0x7fe55c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.519+0000 7fe561508640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 0x7fe55c198750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00d950 tx=0x7fe54c00de10 comp rx=0 tx=0).stop 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 shutdown_connections 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe5480a3da0 0x7fe5480a6260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c10c570 0x7fe55c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c103b50 0x7fe55c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 --2- 172.21.15.37:0/362142737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c103230 0x7fe55c198750 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 >> 172.21.15.37:0/362142737 conn(0x7fe55c077fb0 msgr2=0x7fe55c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 shutdown_connections 2024-04-23T18:26:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.520+0000 7fe561508640 1 -- 172.21.15.37:0/362142737 wait complete. 2024-04-23T18:26:38.526 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id cca99928 --release --await' 2024-04-23T18:26:38.526 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id cca99928 --release --await 2024-04-23T18:26:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 -- 172.21.15.37:0/1931603411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806aef0 msgr2=0x7fbac810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 --2- 172.21.15.37:0/1931603411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806aef0 0x7fbac810e420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009900 tx=0x7fbabc02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 -- 172.21.15.37:0/1931603411 shutdown_connections 2024-04-23T18:26:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 --2- 172.21.15.37:0/1931603411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac810e960 0x7fbac8110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 --2- 172.21.15.37:0/1931603411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806aef0 0x7fbac810e420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 --2- 172.21.15.37:0/1931603411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac806a5d0 0x7fbac806a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 -- 172.21.15.37:0/1931603411 >> 172.21.15.37:0/1931603411 conn(0x7fbac8077eb0 msgr2=0x7fbac80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 -- 172.21.15.37:0/1931603411 shutdown_connections 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.642+0000 7fbacdb20640 1 -- 172.21.15.37:0/1931603411 wait complete. 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 Processor -- start 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 -- start start 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 0x7fbac81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac806aef0 0x7fbac81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac810e960 0x7fbac819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81143d0 con 0x7fbac806a5d0 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8114250 con 0x7fbac806aef0 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbacdb20640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8114550 con 0x7fbac810e960 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbac77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 0x7fbac81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbac77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 0x7fbac81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37254/0 (socket says 172.21.15.37:37254) 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbac77fe640 1 -- 172.21.15.37:0/3076147791 learned_addr learned my addr 172.21.15.37:0/3076147791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbac7fff640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac810e960 0x7fbac819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.643+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac806aef0 0x7fbac81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac77fe640 1 -- 172.21.15.37:0/3076147791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac810e960 msgr2=0x7fbac819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac77fe640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac810e960 0x7fbac819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac77fe640 1 -- 172.21.15.37:0/3076147791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac806aef0 msgr2=0x7fbac81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac77fe640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac806aef0 0x7fbac81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac77fe640 1 -- 172.21.15.37:0/3076147791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac819b030 con 0x7fbac806a5d0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac77fe640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 0x7fbac81a0660 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbab800db80 tx=0x7fbab800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac4ff9640 1 -- 172.21.15.37:0/3076147791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8017070 con 0x7fbac806a5d0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac4ff9640 1 -- 172.21.15.37:0/3076147791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab8003b70 con 0x7fbac806a5d0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac819b320 con 0x7fbac806a5d0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbac4ff9640 1 -- 172.21.15.37:0/3076147791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80106c0 con 0x7fbac806a5d0 2024-04-23T18:26:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.644+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac81a75e0 con 0x7fbac806a5d0 2024-04-23T18:26:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.646+0000 7fbac4ff9640 1 -- 172.21.15.37:0/3076147791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbab8004020 con 0x7fbac806a5d0 2024-04-23T18:26:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.646+0000 7fbad0832740 1 Processor -- start 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 -- start start 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf10870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf1af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf1d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dceae756d0 con 0x55dceaf1b480 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dceae121a0 con 0x55dceaf10490 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbad0832740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dceae71690 con 0x55dceaf10db0 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf1d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf1d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37266/0 (socket says 172.21.15.37:37266) 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 -- 172.21.15.37:0/23359988 learned_addr learned my addr 172.21.15.37:0/23359988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac77fe640 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf1af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac7fff640 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf10870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac4ff9640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbab00a3d20 0x7fbab00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 -- 172.21.15.37:0/23359988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 msgr2=0x55dceaf1af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf1af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 -- 172.21.15.37:0/23359988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 msgr2=0x55dceaf10870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf10870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac6ffd640 1 -- 172.21.15.37:0/23359988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dceae1c5b0 con 0x55dceaf1b480 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac7fff640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbab00a3d20 0x7fbab00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.647+0000 7fbac4ff9640 1 -- 172.21.15.37:0/3076147791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbab80d08e0 con 0x7fbac806a5d0 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf1d870 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009900 tx=0x7fbabc032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dc0fdaebbd674c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbaadffb640 1 -- 172.21.15.37:0/23359988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc03f070 con 0x55dceaf1b480 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbaadffb640 1 -- 172.21.15.37:0/23359988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc0316a0 con 0x55dceaf1b480 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbaadffb640 1 -- 172.21.15.37:0/23359988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc035070 con 0x55dceaf1b480 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 -- 172.21.15.37:0/23359988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 msgr2=0x55dceaf1d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbac7fff640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbab00a3d20 0x7fbab00a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbac806a190 tx=0x7fbac00046c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf1d870 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009900 tx=0x7fbabc032c40 comp rx=0 tx=0).stop 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 -- 172.21.15.37:0/23359988 shutdown_connections 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf1d870 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf1af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 --2- 172.21.15.37:0/23359988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf10870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 -- 172.21.15.37:0/23359988 >> 172.21.15.37:0/23359988 conn(0x55dceaf0c7b0 msgr2=0x55dceaf0cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.648+0000 7fbad0832740 1 -- 172.21.15.37:0/23359988 shutdown_connections 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 -- 172.21.15.37:0/23359988 wait complete. 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 -- start start 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 Processor -- start 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf1ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf18170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf18780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dceaf1e2b0 con 0x55dceaf1b480 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dceae0fb20 con 0x55dceaf10490 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dceaf1e430 con 0x55dceaf10db0 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbad0832740 10 client.0 _ll_register_callbacks cb 0x55dceae0f530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbac7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf18780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbac77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf1ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.649+0000 7fbac7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf18780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37268/0 (socket says 172.21.15.37:37268) 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 -- 172.21.15.37:0/3705655222 learned_addr learned my addr 172.21.15.37:0/3705655222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf18170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 -- 172.21.15.37:0/3705655222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 msgr2=0x55dceaf18170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf18170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 -- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 msgr2=0x55dceaf1ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf1ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 -- 172.21.15.37:0/3705655222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dceab87920 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf18170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbac7fff640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf18780 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbac0012470 tx=0x7fbac0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f62eb3870f7b2be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbaa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/3705655222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac001d860 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/3705655222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbac001da00 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbabc0316a0 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/3705655222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbac001dc00 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbad0832740 10 client.16299 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dceac53d30 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.650+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/3705655222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc0316a0 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/3705655222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fbac001b440 con 0x55dceaf1b480 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbad0832740 10 client.16299 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbad0832740 10 client.16299 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbad0832740 20 client.16299 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbad0832740 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf2b690 0x55dceaf2daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbad0832740 4 client.16299 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dceae33500 con 0x55dceaf2b690 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbac77fe640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf2b690 0x55dceaf2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbac77fe640 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf2b690 0x55dceaf2daf0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbe28759bad8d54 server_cookie=58d45d59d9296bbb in_seq=0 out_seq=0 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.651+0000 7fbaa6ffd640 10 client.16299 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/3705655222 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dceae33500 con 0x55dceaf2b690 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbaa6ffd640 10 client.16299 handle_command_reply: tid=0 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 client.16299 shutdown 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 msgr2=0x55dceaf18780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf18780 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbac0012470 tx=0x7fbac0004730 comp rx=0 tx=0).stop 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 shutdown_connections 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf2b690 0x55dceaf2daf0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf18780 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf18170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbaa6ffd640 0 client.16299 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 --2- 172.21.15.37:0/3705655222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf1ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 >> 172.21.15.37:0/3705655222 conn(0x55dceaf0c7b0 msgr2=0x55dceaf125b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.652+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 shutdown_connections 2024-04-23T18:26:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.653+0000 7fbad0832740 1 -- 172.21.15.37:0/3705655222 wait complete. 2024-04-23T18:26:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.653+0000 7fbad0832740 20 client.16299 trim_cache size 0 max 16384 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 Processor -- start 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 -- start start 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf18360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf12c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81b26e0 con 0x55dceaf1b480 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81aff50 con 0x55dceaf10490 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbad0832740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81adda0 con 0x55dceaf10db0 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbac7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbac7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf170f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37282/0 (socket says 172.21.15.37:37282) 2024-04-23T18:26:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.670+0000 7fbac7fff640 1 -- 172.21.15.37:0/1398782464 learned_addr learned my addr 172.21.15.37:0/1398782464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbac7fff640 1 -- 172.21.15.37:0/1398782464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 msgr2=0x55dceaf12c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbac7fff640 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf12c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbac7fff640 1 -- 172.21.15.37:0/1398782464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 msgr2=0x55dceaf18360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbac7fff640 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf18360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbac7fff640 1 -- 172.21.15.37:0/1398782464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dceaf0d380 con 0x55dceaf1b480 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbac7fff640 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf170f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbac00118b0 tx=0x7fbac00058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2abde7b988a46a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbaad7fa640 1 -- 172.21.15.37:0/1398782464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0033040 con 0x55dceaf1b480 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbaad7fa640 1 -- 172.21.15.37:0/1398782464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac0022030 con 0x55dceaf1b480 2024-04-23T18:26:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.671+0000 7fbaad7fa640 1 -- 172.21.15.37:0/1398782464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0030480 con 0x55dceaf1b480 2024-04-23T18:26:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- 172.21.15.37:0/1398782464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 msgr2=0x55dceaf170f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf170f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbac00118b0 tx=0x7fbac00058f0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- 172.21.15.37:0/1398782464 shutdown_connections 2024-04-23T18:26:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf1b480 0x55dceaf170f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10db0 0x55dceaf12c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- 172.21.15.37:0/1398782464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf10490 0x55dceaf18360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- 172.21.15.37:0/1398782464 >> 172.21.15.37:0/1398782464 conn(0x55dceaf0c7b0 msgr2=0x55dceaf0b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- 172.21.15.37:0/1398782464 shutdown_connections 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- 172.21.15.37:0/1398782464 wait complete. 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- start start 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 Processor -- start 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10490 0x55dceaf23650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 0x55dceaf23b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf1b480 0x55dceaf14420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81ae410 con 0x55dceaf10db0 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81af880 con 0x55dceaf1b480 2024-04-23T18:26:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81aeae0 con 0x55dceaf10490 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.672+0000 7fbad0832740 10 client.0 _ll_register_callbacks cb 0x7fbac0011d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 0x55dceaf23b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 0x55dceaf23b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37294/0 (socket says 172.21.15.37:37294) 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 -- 172.21.15.37:0/1745921292 learned_addr learned my addr 172.21.15.37:0/1745921292 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac7fff640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf1b480 0x55dceaf14420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac77fe640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10490 0x55dceaf23650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 -- 172.21.15.37:0/1745921292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10490 msgr2=0x55dceaf23650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10490 0x55dceaf23650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 -- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf1b480 msgr2=0x55dceaf14420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf1b480 0x55dceaf14420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 -- 172.21.15.37:0/1745921292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dceaf14c30 con 0x55dceaf10db0 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbac6ffd640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 0x55dceaf23b90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x55dceaf0d940 tx=0x7fbabc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cdabc6d85d9436 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbaa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/1745921292 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc035070 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/1745921292 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fbabc031d10 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac0022030 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/1745921292 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbabc052050 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbad0832740 10 client.16305 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.673+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dceaf15f70 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/1745921292 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac0022030 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/1745921292 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fbabc004750 con 0x55dceaf10db0 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbad0832740 10 client.16305 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbad0832740 10 client.16305 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbad0832740 20 client.16305 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbad0832740 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf92c80 0x55dceaf95070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbad0832740 4 client.16305 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cca99928", "release": true, "await": true}] 2024-04-23T18:26:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cca99928", "release": true, "await": true}) -- 0x55dceae830f0 con 0x55dceaf92c80 2024-04-23T18:26:38.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbac77fe640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf92c80 0x55dceaf95070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbaa6ffd640 10 client.16305 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:38.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.674+0000 7fbac77fe640 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf92c80 0x55dceaf95070 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd97e4ce080163ce server_cookie=7a9fb08f29843e02 in_seq=0 out_seq=0 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbaa6ffd640 1 -- 172.21.15.37:0/1745921292 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55dceae830f0 con 0x55dceaf92c80 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbaa6ffd640 10 client.16305 handle_command_reply: tid=0 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 client.16305 shutdown 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 msgr2=0x55dceaf23b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 0x55dceaf23b90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x55dceaf0d940 tx=0x7fbabc004270 comp rx=0 tx=0).stop 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 shutdown_connections 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55dceaf92c80 0x55dceaf95070 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dceaf1b480 0x55dceaf14420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dceaf10db0 0x55dceaf23b90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 --2- 172.21.15.37:0/1745921292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dceaf10490 0x55dceaf23650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbaa6ffd640 0 client.16305 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 >> 172.21.15.37:0/1745921292 conn(0x55dceaf0c7b0 msgr2=0x55dceaf26160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 shutdown_connections 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 1 -- 172.21.15.37:0/1745921292 wait complete. 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.679+0000 7fbad0832740 20 client.16305 trim_cache size 0 max 16384 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 33, 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 24, 2024-04-23T18:26:38.675 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "cca99928": { 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 24, 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.6, 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:26:38.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:26:38.677 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbab00a3d20 msgr2=0x7fbab00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbab00a3d20 0x7fbab00a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbac806a190 tx=0x7fbac00046c0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 msgr2=0x7fbac81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 0x7fbac81a0660 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbab800db80 tx=0x7fbab800b630 comp rx=0 tx=0).stop 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 shutdown_connections 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fbab00a3d20 0x7fbab00a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbac810e960 0x7fbac819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbac806aef0 0x7fbac81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 --2- 172.21.15.37:0/3076147791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbac806a5d0 0x7fbac81a0660 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 >> 172.21.15.37:0/3076147791 conn(0x7fbac8077eb0 msgr2=0x7fbac810fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 shutdown_connections 2024-04-23T18:26:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.680+0000 7fbacdb20640 1 -- 172.21.15.37:0/3076147791 wait complete. 2024-04-23T18:26:38.685 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cca99928', seconds elapsed: 25.7 2024-04-23T18:26:38.685 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-04-23T18:26:38.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2833153066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 msgr2=0x7f0df81035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2833153066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df81035c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009900 tx=0x7f0df402a510 comp rx=0 tx=0).stop 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2833153066 shutdown_connections 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2833153066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df810c520 0x7f0df810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2833153066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df8103b00 0x7f0df810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2833153066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df81035c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2833153066 >> 172.21.15.37:0/2833153066 conn(0x7f0df8077f50 msgr2=0x7f0df8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2833153066 shutdown_connections 2024-04-23T18:26:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.808+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2833153066 wait complete. 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 Processor -- start 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 -- start start 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df8103b00 0x7f0df819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df810c520 0x7f0df8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0df81121b0 con 0x7f0df81031e0 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8112030 con 0x7f0df8103b00 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfeecb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8112330 con 0x7f0df810c520 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37308/0 (socket says 172.21.15.37:37308) 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 -- 172.21.15.37:0/2663641979 learned_addr learned my addr 172.21.15.37:0/2663641979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0deffff640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df8103b00 0x7f0df819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfd468640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df810c520 0x7f0df8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 -- 172.21.15.37:0/2663641979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df810c520 msgr2=0x7f0df8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df810c520 0x7f0df8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 -- 172.21.15.37:0/2663641979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df8103b00 msgr2=0x7f0df819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df8103b00 0x7f0df819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.809+0000 7f0dfcc67640 1 -- 172.21.15.37:0/2663641979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df8198d10 con 0x7f0df81031e0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.810+0000 7f0dfcc67640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df819e3e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005940 tx=0x7f0df40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.810+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df403f030 con 0x7f0df81031e0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.810+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df40044c0 con 0x7f0df81031e0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.810+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df8198fa0 con 0x7f0df81031e0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.810+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4035070 con 0x7f0df81031e0 2024-04-23T18:26:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.810+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df81a52a0 con 0x7f0df81031e0 2024-04-23T18:26:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.811+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df8068d70 con 0x7f0df81031e0 2024-04-23T18:26:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.812+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0df4042030 con 0x7f0df81031e0 2024-04-23T18:26:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.813+0000 7f0dedffb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0de00a3d20 0x7f0de00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.813+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f0df40f0d60 con 0x7f0df81031e0 2024-04-23T18:26:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.813+0000 7f0deffff640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0de00a3d20 0x7f0de00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.813+0000 7f0deffff640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0de00a3d20 0x7f0de00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0df81008f0 tx=0x7f0de8005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:38.817+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0df40bcda0 con 0x7f0df81031e0 2024-04-23T18:26:38.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.33 MB/sec execute 163 sec latency 25114.939 ms 2024-04-23T18:26:39.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.041+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0df81076a0 con 0x7f0df81031e0 2024-04-23T18:26:39.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.042+0000 7f0dedffb640 1 -- 172.21.15.37:0/2663641979 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11879 (secure 0 0 0) 0x7f0df40c10a0 con 0x7f0df81031e0 2024-04-23T18:26:39.036 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:26:39.036 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:26:17.565566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24484,24509,24479]},"id":1}]} 2024-04-23T18:26:39.036 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0de00a3d20 msgr2=0x7f0de00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0de00a3d20 0x7f0de00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0df81008f0 tx=0x7f0de8005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 msgr2=0x7f0df819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df819e3e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005940 tx=0x7f0df40040c0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 shutdown_connections 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f0de00a3d20 0x7f0de00a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df810c520 0x7f0df8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0df8103b00 0x7f0df819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 --2- 172.21.15.37:0/2663641979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df81031e0 0x7f0df819e3e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 >> 172.21.15.37:0/2663641979 conn(0x7f0df8077f50 msgr2=0x7f0df80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 shutdown_connections 2024-04-23T18:26:39.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.044+0000 7f0dfeecb640 1 -- 172.21.15.37:0/2663641979 wait complete. 2024-04-23T18:26:39.048 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 84.6 --await-for 42.0' 2024-04-23T18:26:39.049 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 84.6 --await-for 42.0 2024-04-23T18:26:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.168+0000 7f93c487c640 1 -- 172.21.15.37:0/1908776548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 msgr2=0x7f93c010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.168+0000 7f93c487c640 1 --2- 172.21.15.37:0/1908776548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 0x7f93c010dd40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009900 tx=0x7f93b802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 -- 172.21.15.37:0/1908776548 shutdown_connections 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 --2- 172.21.15.37:0/1908776548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 0x7f93c010dd40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 --2- 172.21.15.37:0/1908776548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c0101a50 0x7f93c0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 --2- 172.21.15.37:0/1908776548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93c01010a0 0x7f93c0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 -- 172.21.15.37:0/1908776548 >> 172.21.15.37:0/1908776548 conn(0x7f93c0077f50 msgr2=0x7f93c0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 -- 172.21.15.37:0/1908776548 shutdown_connections 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 -- 172.21.15.37:0/1908776548 wait complete. 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 Processor -- start 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.169+0000 7f93c487c640 1 -- start start 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93c01010a0 0x7f93c01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 0x7f93c0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c019cfa0 0x7f93c019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01110e0 con 0x7f93c0106170 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93c0110f60 con 0x7f93c019cfa0 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93c0111260 con 0x7f93c01010a0 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 0x7f93c0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 0x7f93c0198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37320/0 (socket says 172.21.15.37:37320) 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2683414709 learned_addr learned my addr 172.21.15.37:0/2683414709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93be59c640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93c01010a0 0x7f93c01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bed9d640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c019cfa0 0x7f93c019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2683414709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93c01010a0 msgr2=0x7f93c01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93c01010a0 0x7f93c01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2683414709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c019cfa0 msgr2=0x7f93c019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93c019cfa0 0x7f93c019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2683414709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c019da90 con 0x7f93c0106170 2024-04-23T18:26:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93c0106170 0x7f93c0198c20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f93b400d770 tx=0x7f93b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93af7fe640 1 -- 172.21.15.37:0/2683414709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4017070 con 0x7f93c0106170 2024-04-23T18:26:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93c487c640 1 -- 172.21.15.37:0/2683414709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c019dd80 con 0x7f93c0106170 2024-04-23T18:26:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.170+0000 7f93af7fe640 1 -- 172.21.15.37:0/2683414709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b4009c30 con 0x7f93c0106170 2024-04-23T18:26:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.171+0000 7f93c487c640 1 -- 172.21.15.37:0/2683414709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c01a55c0 con 0x7f93c0106170 2024-04-23T18:26:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.171+0000 7f93af7fe640 1 -- 172.21.15.37:0/2683414709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b400f620 con 0x7f93c0106170 2024-04-23T18:26:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.173+0000 7f93af7fe640 1 -- 172.21.15.37:0/2683414709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93b401a070 con 0x7f93c0106170 2024-04-23T18:26:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.173+0000 7f93c7597740 1 Processor -- start 2024-04-23T18:26:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 -- start start 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784032380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784040f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784043890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556783f97370 con 0x5567840328c0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556783f342e0 con 0x5567840414a0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93c7597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556783f93330 con 0x556784031fa0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784040f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784040f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37322/0 (socket says 172.21.15.37:37322) 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 -- 172.21.15.37:0/1063418902 learned_addr learned my addr 172.21.15.37:0/1063418902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93af7fe640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93a80a3da0 0x7f93a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784043890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93bed9d640 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784032380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93bed9d640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93a80a3da0 0x7f93a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 -- 172.21.15.37:0/1063418902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 msgr2=0x556784032380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784032380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 -- 172.21.15.37:0/1063418902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 msgr2=0x556784043890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784043890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93be59c640 1 -- 172.21.15.37:0/1063418902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556783d75d80 con 0x5567840328c0 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.174+0000 7f93af7fe640 1 -- 172.21.15.37:0/2683414709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f93b40d0110 con 0x7f93c0106170 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93be59c640 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784040f60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f93b00114f0 tx=0x7f93b00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2990ab878b98999a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93acff9640 1 -- 172.21.15.37:0/1063418902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004d20 con 0x5567840328c0 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93bed9d640 1 --2- 172.21.15.37:0/2683414709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93a80a3da0 0x7f93a80a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006010 tx=0x7f93b80058d0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93acff9640 1 -- 172.21.15.37:0/1063418902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b0004050 con 0x5567840328c0 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93acff9640 1 -- 172.21.15.37:0/1063418902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b000f7c0 con 0x5567840328c0 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93c7597740 1 -- 172.21.15.37:0/1063418902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 msgr2=0x556784040f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.175+0000 7f93c7597740 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784040f60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f93b00114f0 tx=0x7f93b00119b0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- 172.21.15.37:0/1063418902 shutdown_connections 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784043890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784040f60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 --2- 172.21.15.37:0/1063418902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784032380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- 172.21.15.37:0/1063418902 >> 172.21.15.37:0/1063418902 conn(0x55678402e3c0 msgr2=0x55678402e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- 172.21.15.37:0/1063418902 shutdown_connections 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- 172.21.15.37:0/1063418902 wait complete. 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- start start 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 Processor -- start 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556784031fa0 0x55678403e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678403e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567840414a0 0x55678403ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556784044260 con 0x5567840328c0 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556783f31fb0 con 0x556784031fa0 2024-04-23T18:26:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5567840443e0 con 0x5567840414a0 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678403e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93c7597740 10 client.0 _ll_register_callbacks cb 0x556783f319a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678403e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37336/0 (socket says 172.21.15.37:37336) 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.176+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1733411228 learned_addr learned my addr 172.21.15.37:0/1733411228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93be59c640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556784031fa0 0x55678403e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bed9d640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567840414a0 0x55678403ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1733411228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567840414a0 msgr2=0x55678403ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567840414a0 0x55678403ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556784031fa0 msgr2=0x55678403e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556784031fa0 0x55678403e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1733411228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556783dd6510 con 0x5567840328c0 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93be59c640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556784031fa0 0x55678403e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678403e8f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f93b4098bc0 tx=0x7f93b409bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eeb2683c4bc654 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93a5ffb640 1 -- 172.21.15.37:0/1733411228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4021050 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b0004050 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93a5ffb640 1 -- 172.21.15.37:0/1733411228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f93b40a01d0 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93a5ffb640 1 -- 172.21.15.37:0/1733411228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93b40a04f0 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93c7597740 10 client.16323 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556783c69670 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.177+0000 7f93a5ffb640 1 -- 172.21.15.37:0/1733411228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b0004050 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93a5ffb640 1 -- 172.21.15.37:0/1733411228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f93b408bbd0 con 0x5567840328c0 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93c7597740 10 client.16323 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93c7597740 10 client.16323 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93c7597740 20 client.16323 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93c7597740 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840517a0 0x556784053c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93c7597740 4 client.16323 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556783fa4d90 con 0x5567840517a0 2024-04-23T18:26:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93be59c640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840517a0 0x556784053c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93be59c640 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840517a0 0x556784053c30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdcc24c4f3fbc0f server_cookie=d1047e0eb7d14294 in_seq=0 out_seq=0 2024-04-23T18:26:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.178+0000 7f93a5ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93a5ffb640 1 -- 172.21.15.37:0/1733411228 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556783fa4d90 con 0x5567840517a0 2024-04-23T18:26:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93a5ffb640 10 client.16323 handle_command_reply: tid=0 2024-04-23T18:26:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 client.16323 shutdown 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 msgr2=0x55678403e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678403e8f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f93b4098bc0 tx=0x7f93b409bb50 comp rx=0 tx=0).stop 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 shutdown_connections 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840517a0 0x556784053c30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5567840414a0 0x55678403ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93a5ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678403e8f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 --2- 172.21.15.37:0/1733411228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556784031fa0 0x55678403e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 >> 172.21.15.37:0/1733411228 conn(0x55678402e3c0 msgr2=0x5567840341f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 shutdown_connections 2024-04-23T18:26:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.179+0000 7f93c7597740 1 -- 172.21.15.37:0/1733411228 wait complete. 2024-04-23T18:26:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.180+0000 7f93c7597740 20 client.16323 trim_cache size 0 max 16384 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 Processor -- start 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 -- start start 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x55678404d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678402dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784059540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01ae330 con 0x5567840328c0 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01abba0 con 0x5567840414a0 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93c7597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01a99f0 con 0x556784031fa0 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678402dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678402dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37340/0 (socket says 172.21.15.37:37340) 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1827409150 learned_addr learned my addr 172.21.15.37:0/1827409150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1827409150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 msgr2=0x55678404d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x55678404d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1827409150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 msgr2=0x556784059540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784059540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.197+0000 7f93bdd9b640 1 -- 172.21.15.37:0/1827409150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784032e50 con 0x5567840328c0 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678402dd80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f93b40b0fd0 tx=0x7f93b400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab153cf1526dc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93a7fff640 1 -- 172.21.15.37:0/1827409150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4021040 con 0x5567840328c0 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93a7fff640 1 -- 172.21.15.37:0/1827409150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b40b5070 con 0x5567840328c0 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93a7fff640 1 -- 172.21.15.37:0/1827409150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40982b0 con 0x5567840328c0 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 -- 172.21.15.37:0/1827409150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 msgr2=0x55678402dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678402dd80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f93b40b0fd0 tx=0x7f93b400df10 comp rx=0 tx=0).stop 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 -- 172.21.15.37:0/1827409150 shutdown_connections 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784059540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x55678402dd80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 --2- 172.21.15.37:0/1827409150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x55678404d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 -- 172.21.15.37:0/1827409150 >> 172.21.15.37:0/1827409150 conn(0x55678402e3c0 msgr2=0x556784040470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 -- 172.21.15.37:0/1827409150 shutdown_connections 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.198+0000 7f93c7597740 1 -- 172.21.15.37:0/1827409150 wait complete. 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 -- start start 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 Processor -- start 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784049670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784049bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784046640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01aa060 con 0x5567840328c0 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01ab4d0 con 0x5567840414a0 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93c01aa730 con 0x556784031fa0 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93c7597740 10 client.0 _ll_register_callbacks cb 0x7f93b409bfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784049bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784049bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37350/0 (socket says 172.21.15.37:37350) 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2124778938 learned_addr learned my addr 172.21.15.37:0/2124778938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93be59c640 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784049670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2124778938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 msgr2=0x556784049670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784049670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 msgr2=0x556784046640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784046640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 -- 172.21.15.37:0/2124778938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556784046c00 con 0x5567840328c0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93bdd9b640 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784049bb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009610 tx=0x7f93b409c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942f63aa15b530af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.199+0000 7f93a5ffb640 1 -- 172.21.15.37:0/2124778938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4021040 con 0x5567840328c0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93a5ffb640 1 -- 172.21.15.37:0/2124778938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f93b4098240 con 0x5567840328c0 2024-04-23T18:26:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b40b5070 con 0x5567840328c0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93a5ffb640 1 -- 172.21.15.37:0/2124778938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93b40a4db0 con 0x5567840328c0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 10 client.16329 fetch_fsmap learned FSMap version 33 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567840b8620 con 0x5567840328c0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93a5ffb640 1 -- 172.21.15.37:0/2124778938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b40b5070 con 0x5567840328c0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93a5ffb640 1 -- 172.21.15.37:0/2124778938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f93b40a1880 con 0x5567840328c0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 10 client.16329 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 10 client.16329 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 20 client.16329 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840b8bf0 0x5567840bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 4 client.16329 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.6, "await_for": 42.0}] 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.6, "await_for": 42.0}) -- 0x556783f9ec00 con 0x5567840b8bf0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.200+0000 7f93be59c640 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840b8bf0 0x5567840bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.201+0000 7f93a5ffb640 10 client.16329 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:26:39.201+0000 7f93be59c640 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840b8bf0 0x5567840bb010 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe4113b04c0f9f9 server_cookie=3f63c5c2610a760b in_seq=0 out_seq=0 2024-04-23T18:26:39.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/362142737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:39 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2663641979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/362142737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:39 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2663641979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/362142737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:39 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2663641979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.825 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.30 MB/sec execute 164 sec latency 1034.491 ms 2024-04-23T18:26:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:40 smithi172 ceph-mon[32456]: pgmap v190: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-04-23T18:26:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:40 smithi173 ceph-mon[32906]: pgmap v190: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-04-23T18:26:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:40 smithi037 ceph-mon[29404]: pgmap v190: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-04-23T18:26:40.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.28 MB/sec execute 165 sec latency 2034.612 ms 2024-04-23T18:26:41.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:41 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:41.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:41 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:41.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:41 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:41.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.25 MB/sec execute 166 sec latency 3034.701 ms 2024-04-23T18:26:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:42 smithi172 ceph-mon[32456]: pgmap v191: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-04-23T18:26:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:42 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:42 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:42 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:26:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:42 smithi173 ceph-mon[32906]: pgmap v191: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-04-23T18:26:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:42 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:42 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:42 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:26:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:42 smithi037 ceph-mon[29404]: pgmap v191: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-04-23T18:26:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:42 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:42 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:42.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:42 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:26:42.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.22 MB/sec execute 167 sec latency 4034.807 ms 2024-04-23T18:26:43.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.20 MB/sec execute 168 sec latency 5034.891 ms 2024-04-23T18:26:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:44 smithi172 ceph-mon[32456]: pgmap v192: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-04-23T18:26:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:44 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:26:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:44 smithi173 ceph-mon[32906]: pgmap v192: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-04-23T18:26:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:44 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:26:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:44 smithi037 ceph-mon[29404]: pgmap v192: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-04-23T18:26:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:44 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:26:44.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.17 MB/sec execute 169 sec latency 6035.008 ms 2024-04-23T18:26:45.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.15 MB/sec execute 170 sec latency 7035.116 ms 2024-04-23T18:26:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:46 smithi172 ceph-mon[32456]: pgmap v193: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-23T18:26:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:46 smithi173 ceph-mon[32906]: pgmap v193: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-23T18:26:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:46 smithi037 ceph-mon[29404]: pgmap v193: 97 pgs: 97 active+clean; 199 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-23T18:26:46.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.13 MB/sec execute 171 sec latency 8035.242 ms 2024-04-23T18:26:47.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.10 MB/sec execute 172 sec latency 9035.364 ms 2024-04-23T18:26:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:48 smithi173 ceph-mon[32906]: pgmap v194: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-04-23T18:26:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:48 smithi037 ceph-mon[29404]: pgmap v194: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-04-23T18:26:48.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:48 smithi172 ceph-mon[32456]: pgmap v194: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-04-23T18:26:48.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.08 MB/sec execute 173 sec latency 10035.478 ms 2024-04-23T18:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 -- 172.21.15.37:0/946747155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103220 msgr2=0x7fe964103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 --2- 172.21.15.37:0/946747155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103220 0x7fe964103600 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe954009ca0 tx=0x7fe95402a950 comp rx=0 tx=0).stop 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 -- 172.21.15.37:0/946747155 shutdown_connections 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 --2- 172.21.15.37:0/946747155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 0x7fe96410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 --2- 172.21.15.37:0/946747155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe964103b40 0x7fe96410c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 --2- 172.21.15.37:0/946747155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103220 0x7fe964103600 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 -- 172.21.15.37:0/946747155 >> 172.21.15.37:0/946747155 conn(0x7fe964077fd0 msgr2=0x7fe9640783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 -- 172.21.15.37:0/946747155 shutdown_connections 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.633+0000 7fe968fa8640 1 -- 172.21.15.37:0/946747155 wait complete. 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 Processor -- start 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 -- start start 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe964103220 0x7fe964198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 0x7fe964198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 0x7fe96419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641121b0 con 0x7fe964103b40 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe964112030 con 0x7fe96410c560 2024-04-23T18:26:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe968fa8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe964112330 con 0x7fe964103220 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 0x7fe964198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 0x7fe964198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48532/0 (socket says 172.21.15.37:48532) 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 -- 172.21.15.37:0/1148802493 learned_addr learned my addr 172.21.15.37:0/1148802493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe96259c640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe964103220 0x7fe964198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 0x7fe96419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 -- 172.21.15.37:0/1148802493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe964103220 msgr2=0x7fe964198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe964103220 0x7fe964198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 -- 172.21.15.37:0/1148802493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 msgr2=0x7fe96419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 0x7fe96419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe961d9b640 1 -- 172.21.15.37:0/1148802493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96419d710 con 0x7fe964103b40 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.634+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 0x7fe96419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.635+0000 7fe961d9b640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 0x7fe964198cb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d770 tx=0x7fe95800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.635+0000 7fe9537fe640 1 -- 172.21.15.37:0/1148802493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958017070 con 0x7fe964103b40 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.635+0000 7fe9537fe640 1 -- 172.21.15.37:0/1148802493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9580054d0 con 0x7fe964103b40 2024-04-23T18:26:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.635+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96419da00 con 0x7fe964103b40 2024-04-23T18:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.635+0000 7fe9537fe640 1 -- 172.21.15.37:0/1148802493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958004c30 con 0x7fe964103b40 2024-04-23T18:26:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.635+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641a5290 con 0x7fe964103b40 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.637+0000 7fe9537fe640 1 -- 172.21.15.37:0/1148802493 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe958004e10 con 0x7fe964103b40 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.637+0000 7fe96bd95740 1 Processor -- start 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 -- start start 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddaf8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddb032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556edda5d970 con 0x556eddaf8620 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556edd9fa4a0 con 0x556eddaf8f40 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96bd95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556edda59930 con 0x556eddb03800 2024-04-23T18:26:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe9537fe640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe94c0a3dc0 0x7fe94c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe962d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddaf8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe962d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddaf8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48536/0 (socket says 172.21.15.37:48536) 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe962d9d640 1 -- 172.21.15.37:0/1510457231 learned_addr learned my addr 172.21.15.37:0/1510457231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe94c0a3dc0 0x7fe94c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe961d9b640 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe96259c640 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddb032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.638+0000 7fe9537fe640 1 -- 172.21.15.37:0/1148802493 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe9580d02e0 con 0x7fe964103b40 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 -- 172.21.15.37:0/1510457231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 msgr2=0x556eddb05bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb05bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 -- 172.21.15.37:0/1510457231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 msgr2=0x556eddb032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddb032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 -- 172.21.15.37:0/1510457231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556edd770210 con 0x556eddaf8620 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddaf8a00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe95c012a30 tx=0x7fe95c012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f24f610cabf95d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe962d9d640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe94c0a3dc0 0x7fe94c0a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe95c010860 tx=0x7fe95c01d000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe950ff9640 1 -- 172.21.15.37:0/1510457231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c023070 con 0x556eddaf8620 2024-04-23T18:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe950ff9640 1 -- 172.21.15.37:0/1510457231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c009990 con 0x556eddaf8620 2024-04-23T18:26:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.639+0000 7fe950ff9640 1 -- 172.21.15.37:0/1510457231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00da20 con 0x556eddaf8620 2024-04-23T18:26:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 -- 172.21.15.37:0/1510457231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 msgr2=0x556eddaf8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddaf8a00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe95c012a30 tx=0x7fe95c012ef0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 -- 172.21.15.37:0/1510457231 shutdown_connections 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb05bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddb032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 --2- 172.21.15.37:0/1510457231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddaf8a00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 -- 172.21.15.37:0/1510457231 >> 172.21.15.37:0/1510457231 conn(0x556eddaf4b30 msgr2=0x556eddaf4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 -- 172.21.15.37:0/1510457231 shutdown_connections 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 -- 172.21.15.37:0/1510457231 wait complete. 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 -- start start 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.640+0000 7fe96bd95740 1 Processor -- start 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddb031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 0x556eddb004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb00a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556eddb063d0 con 0x556eddaf8f40 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556edd9f80b0 con 0x556eddaf8620 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556eddb06550 con 0x556eddb03800 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96bd95740 10 client.0 _ll_register_callbacks cb 0x556edd9e5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 0x556eddb004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 0x556eddb004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48544/0 (socket says 172.21.15.37:48544) 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 -- 172.21.15.37:0/2915193808 learned_addr learned my addr 172.21.15.37:0/2915193808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe962d9d640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb00a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe96259c640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddb031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 -- 172.21.15.37:0/2915193808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 msgr2=0x556eddb00a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb00a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 -- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 msgr2=0x556eddb031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddb031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 -- 172.21.15.37:0/2915193808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556edd83c620 con 0x556eddaf8f40 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 0x556eddb004a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe958009610 tx=0x7fe9580209a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f799836e20a517e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.641+0000 7fe949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe949ffb640 1 -- 172.21.15.37:0/2915193808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580cf040 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe949ffb640 1 -- 172.21.15.37:0/2915193808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe9580cdca0 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95c009990 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe949ffb640 1 -- 172.21.15.37:0/2915193808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe95809bbd0 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 10 client.16341 fetch_fsmap learned FSMap version 34 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556eddb123a0 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe949ffb640 1 -- 172.21.15.37:0/2915193808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c009990 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe949ffb640 1 -- 172.21.15.37:0/2915193808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fe9580988e0 con 0x556eddaf8f40 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 10 client.16341 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 20 client.16341 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb13810 0x556eddb15c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 4 client.16341 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.642+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556edda6b390 con 0x556eddb13810 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.643+0000 7fe96259c640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb13810 0x556eddb15c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.643+0000 7fe949ffb640 10 client.16341 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.643+0000 7fe96259c640 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb13810 0x556eddb15c60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27039e9d2136dbb3 server_cookie=88ad2020197e00fd in_seq=0 out_seq=0 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe949ffb640 1 -- 172.21.15.37:0/2915193808 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556edda6b390 con 0x556eddb13810 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe949ffb640 10 client.16341 handle_command_reply: tid=0 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 client.16341 shutdown 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 msgr2=0x556eddb004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 0x556eddb004a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe958009610 tx=0x7fe9580209a0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 shutdown_connections 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb13810 0x556eddb15c60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb00a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe949ffb640 0 client.16341 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8f40 0x556eddb004a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2915193808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddb031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 >> 172.21.15.37:0/2915193808 conn(0x556eddaf4b30 msgr2=0x556eddafa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 shutdown_connections 2024-04-23T18:26:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 1 -- 172.21.15.37:0/2915193808 wait complete. 2024-04-23T18:26:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.644+0000 7fe96bd95740 20 client.16341 trim_cache size 0 max 16384 2024-04-23T18:26:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.659+0000 7fe96bd95740 1 Processor -- start 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.659+0000 7fe96bd95740 1 -- start start 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddb0f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddaf4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb02ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96bd95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b04d0 con 0x556eddaf8620 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96bd95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641add40 con 0x556eddaf8f40 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96bd95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641abb90 con 0x556eddb03800 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddb0f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddb0f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48556/0 (socket says 172.21.15.37:48556) 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 -- 172.21.15.37:0/2144804900 learned_addr learned my addr 172.21.15.37:0/2144804900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddaf4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 -- 172.21.15.37:0/2144804900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 msgr2=0x556eddb02ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb02ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 -- 172.21.15.37:0/2144804900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 msgr2=0x556eddaf4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddaf4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 -- 172.21.15.37:0/2144804900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eddaff450 con 0x556eddaf8620 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddaf4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe96259c640 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddb0f290 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe954009600 tx=0x7fe95402f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582558fa975548ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe94bfff640 1 -- 172.21.15.37:0/2144804900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954034040 con 0x556eddaf8620 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe94bfff640 1 -- 172.21.15.37:0/2144804900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe954003e20 con 0x556eddaf8620 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.660+0000 7fe94bfff640 1 -- 172.21.15.37:0/2144804900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954004610 con 0x556eddaf8620 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144804900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 msgr2=0x556eddb0f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddb0f290 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe954009600 tx=0x7fe95402f670 comp rx=0 tx=0).stop 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144804900 shutdown_connections 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddb03800 0x556eddb02ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8f40 0x556eddaf4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144804900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddaf8620 0x556eddb0f290 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144804900 >> 172.21.15.37:0/2144804900 conn(0x556eddaf4b30 msgr2=0x556eddafbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144804900 shutdown_connections 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144804900 wait complete. 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 -- start start 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.661+0000 7fe96bd95740 1 Processor -- start 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddafd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddaf8f40 0x556eddafd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 0x556eddb7d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641ac200 con 0x556eddb03800 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641ad670 con 0x556eddaf8620 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641ac8d0 con 0x556eddaf8f40 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 10 client.0 _ll_register_callbacks cb 0x7fe958098510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 0x556eddb7d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 0x556eddb7d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48566/0 (socket says 172.21.15.37:48566) 2024-04-23T18:26:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 -- 172.21.15.37:0/2144853791 learned_addr learned my addr 172.21.15.37:0/2144853791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96259c640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddafd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddaf8f40 0x556eddafd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 -- 172.21.15.37:0/2144853791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddaf8f40 msgr2=0x556eddafd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddaf8f40 0x556eddafd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 -- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 msgr2=0x556eddafd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddafd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 -- 172.21.15.37:0/2144853791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eddafe280 con 0x556eddb03800 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe961d9b640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddaf8f40 0x556eddafd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe962d9d640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 0x556eddb7d1f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0057c0 tx=0x7fe95c009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5ff3518142d0b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe949ffb640 1 -- 172.21.15.37:0/2144853791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c023040 con 0x556eddb03800 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe949ffb640 1 -- 172.21.15.37:0/2144853791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe95c012570 con 0x556eddb03800 2024-04-23T18:26:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.662+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe954003e20 con 0x556eddb03800 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe949ffb640 1 -- 172.21.15.37:0/2144853791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe95c00db40 con 0x556eddb03800 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 10 client.16347 fetch_fsmap learned FSMap version 34 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556eddb7e780 con 0x556eddb03800 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe949ffb640 1 -- 172.21.15.37:0/2144853791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe954003e20 con 0x556eddb03800 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe949ffb640 1 -- 172.21.15.37:0/2144853791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fe95c02e020 con 0x556eddb03800 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 10 client.16347 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 20 client.16347 populate_metadata read hostname 'smithi037' 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb7ed60 0x556eddb81180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 4 client.16347 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556edda65200 con 0x556eddb7ed60 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.663+0000 7fe96259c640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb7ed60 0x556eddb81180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.664+0000 7fe96259c640 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb7ed60 0x556eddb81180 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8c132c8250c521 server_cookie=1cfe9046256a2746 in_seq=0 out_seq=0 2024-04-23T18:26:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:26:49.664+0000 7fe949ffb640 10 client.16347 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:26:49.826 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.05 MB/sec execute 174 sec latency 11035.541 ms 2024-04-23T18:26:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:50 smithi173 ceph-mon[32906]: pgmap v195: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-04-23T18:26:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:50 smithi037 ceph-mon[29404]: pgmap v195: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-04-23T18:26:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:50 smithi172 ceph-mon[32456]: pgmap v195: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-04-23T18:26:50.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.03 MB/sec execute 175 sec latency 12035.683 ms 2024-04-23T18:26:51.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 4.01 MB/sec execute 176 sec latency 13035.785 ms 2024-04-23T18:26:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:52 smithi173 ceph-mon[32906]: pgmap v196: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-04-23T18:26:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:52 smithi037 ceph-mon[29404]: pgmap v196: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-04-23T18:26:52.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:52 smithi172 ceph-mon[32456]: pgmap v196: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-04-23T18:26:52.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.99 MB/sec execute 177 sec latency 14035.889 ms 2024-04-23T18:26:53.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.96 MB/sec execute 178 sec latency 15035.952 ms 2024-04-23T18:26:54.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:54 smithi173 ceph-mon[32906]: pgmap v197: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-23T18:26:54.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:54 smithi037 ceph-mon[29404]: pgmap v197: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-23T18:26:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:54 smithi172 ceph-mon[32456]: pgmap v197: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-23T18:26:54.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.94 MB/sec execute 179 sec latency 16036.059 ms 2024-04-23T18:26:55.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.92 MB/sec execute 180 sec latency 17036.198 ms 2024-04-23T18:26:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:56 smithi173 ceph-mon[32906]: pgmap v198: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 769 KiB/s wr, 39 op/s 2024-04-23T18:26:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:56 smithi037 ceph-mon[29404]: pgmap v198: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 769 KiB/s wr, 39 op/s 2024-04-23T18:26:56.389 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:56 smithi172 ceph-mon[32456]: pgmap v198: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 769 KiB/s wr, 39 op/s 2024-04-23T18:26:56.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.90 MB/sec execute 181 sec latency 18036.237 ms 2024-04-23T18:26:57.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.88 MB/sec execute 182 sec latency 19036.274 ms 2024-04-23T18:26:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:26:58 smithi173 ceph-mon[32906]: pgmap v199: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 772 KiB/s wr, 43 op/s 2024-04-23T18:26:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:26:58 smithi037 ceph-mon[29404]: pgmap v199: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 772 KiB/s wr, 43 op/s 2024-04-23T18:26:58.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:26:58 smithi172 ceph-mon[32456]: pgmap v199: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 772 KiB/s wr, 43 op/s 2024-04-23T18:26:58.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.86 MB/sec execute 183 sec latency 20036.312 ms 2024-04-23T18:26:59.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.83 MB/sec execute 184 sec latency 21036.406 ms 2024-04-23T18:27:00.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:00 smithi037 ceph-mon[29404]: pgmap v200: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:27:00.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:00 smithi172 ceph-mon[32456]: pgmap v200: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:27:00.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:00 smithi173 ceph-mon[32906]: pgmap v200: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:27:00.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.81 MB/sec execute 185 sec latency 22036.529 ms 2024-04-23T18:27:01.827 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.79 MB/sec execute 186 sec latency 23036.604 ms 2024-04-23T18:27:02.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:02 smithi037 ceph-mon[29404]: pgmap v201: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:27:02.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:02 smithi172 ceph-mon[32456]: pgmap v201: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:27:02.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:02 smithi173 ceph-mon[32906]: pgmap v201: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:27:02.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.77 MB/sec execute 187 sec latency 24036.721 ms 2024-04-23T18:27:03.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.75 MB/sec execute 188 sec latency 25036.827 ms 2024-04-23T18:27:04.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:04 smithi037 ceph-mon[29404]: pgmap v202: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-23T18:27:04.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:04 smithi172 ceph-mon[32456]: pgmap v202: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-23T18:27:04.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:04 smithi173 ceph-mon[32906]: pgmap v202: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-23T18:27:04.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.73 MB/sec execute 189 sec latency 26036.921 ms 2024-04-23T18:27:05.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.71 MB/sec execute 190 sec latency 27037.018 ms 2024-04-23T18:27:06.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:06 smithi037 ceph-mon[29404]: pgmap v203: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-04-23T18:27:06.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:06 smithi172 ceph-mon[32456]: pgmap v203: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-04-23T18:27:06.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:06 smithi173 ceph-mon[32906]: pgmap v203: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-04-23T18:27:06.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.69 MB/sec execute 191 sec latency 28037.127 ms 2024-04-23T18:27:07.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.67 MB/sec execute 192 sec latency 29037.245 ms 2024-04-23T18:27:08.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:08 smithi172 ceph-mon[32456]: pgmap v204: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-23T18:27:08.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:08 smithi173 ceph-mon[32906]: pgmap v204: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-23T18:27:08.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:08 smithi037 ceph-mon[29404]: pgmap v204: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-04-23T18:27:08.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.66 MB/sec execute 193 sec latency 30037.399 ms 2024-04-23T18:27:09.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.64 MB/sec execute 194 sec latency 31037.502 ms 2024-04-23T18:27:10.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:10 smithi172 ceph-mon[32456]: pgmap v205: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-23T18:27:10.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:10 smithi173 ceph-mon[32906]: pgmap v205: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-23T18:27:10.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:10 smithi037 ceph-mon[29404]: pgmap v205: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-23T18:27:10.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.62 MB/sec execute 195 sec latency 32037.578 ms 2024-04-23T18:27:11.828 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.60 MB/sec execute 196 sec latency 33037.663 ms 2024-04-23T18:27:12.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:12 smithi172 ceph-mon[32456]: pgmap v206: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-23T18:27:12.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:12 smithi173 ceph-mon[32906]: pgmap v206: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-23T18:27:12.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:12 smithi037 ceph-mon[29404]: pgmap v206: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-04-23T18:27:12.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.58 MB/sec execute 197 sec latency 34037.762 ms 2024-04-23T18:27:13.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.56 MB/sec execute 198 sec latency 35037.894 ms 2024-04-23T18:27:14.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:14 smithi172 ceph-mon[32456]: pgmap v207: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-04-23T18:27:14.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:14 smithi173 ceph-mon[32906]: pgmap v207: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-04-23T18:27:14.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:14 smithi037 ceph-mon[29404]: pgmap v207: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-04-23T18:27:14.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.55 MB/sec execute 199 sec latency 36037.964 ms 2024-04-23T18:27:15.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.53 MB/sec execute 200 sec latency 37038.002 ms 2024-04-23T18:27:16.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:16 smithi172 ceph-mon[32456]: pgmap v208: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-23T18:27:16.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:16 smithi173 ceph-mon[32906]: pgmap v208: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-23T18:27:16.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:16 smithi037 ceph-mon[29404]: pgmap v208: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-23T18:27:16.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.51 MB/sec execute 201 sec latency 38038.145 ms 2024-04-23T18:27:17.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.49 MB/sec execute 202 sec latency 39038.244 ms 2024-04-23T18:27:18.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:18 smithi172 ceph-mon[32456]: pgmap v209: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T18:27:18.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:18 smithi173 ceph-mon[32906]: pgmap v209: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T18:27:18.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:18 smithi037 ceph-mon[29404]: pgmap v209: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T18:27:18.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.48 MB/sec execute 203 sec latency 40038.368 ms 2024-04-23T18:27:19.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.46 MB/sec execute 204 sec latency 41038.428 ms 2024-04-23T18:27:20.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:20 smithi172 ceph-mon[32456]: pgmap v210: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-23T18:27:20.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:20 smithi173 ceph-mon[32906]: pgmap v210: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-23T18:27:20.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:20 smithi037 ceph-mon[29404]: pgmap v210: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-23T18:27:20.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.44 MB/sec execute 205 sec latency 42038.532 ms 2024-04-23T18:27:21.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93a5ffb640 1 -- 172.21.15.37:0/2124778938 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: -115 ) ==== 8+0+594 (crc 0 0 0) 0x556783f9ec00 con 0x5567840b8bf0 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93a5ffb640 10 client.16329 handle_command_reply: tid=0 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 client.16329 shutdown 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 msgr2=0x556784049bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784049bb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009610 tx=0x7f93b409c370 comp rx=0 tx=0).stop 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 shutdown_connections 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5567840b8bf0 0x5567840bb010 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5567840414a0 0x556784046640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567840328c0 0x556784049bb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93a5ffb640 0 client.16329 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 --2- 172.21.15.37:0/2124778938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556784031fa0 0x556784049670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 >> 172.21.15.37:0/2124778938 conn(0x55678402e3c0 msgr2=0x55678403e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 shutdown_connections 2024-04-23T18:27:21.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.203+0000 7f93c7597740 1 -- 172.21.15.37:0/2124778938 wait complete. 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.204+0000 7f93c7597740 20 client.16329 trim_cache size 0 max 16384 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 34, 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 25, 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "9f848727": { 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 25, 2024-04-23T18:27:21.198 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCING", 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.0 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.6, 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCING", 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.0 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:21.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:21.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:21.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:21.200 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:27:21.200 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-04-23T18:27:21.201 INFO:teuthology.orchestra.run.smithi037.stderr:Error EINPROGRESS: 2024-04-23T18:27:21.201 WARNING:tasks.quiescer.fs.[cephfs]:Set '9f848727' hasn't quiesced after 42.5 seconds (timeout: 60). Dumping ops with locks from all ranks. 2024-04-23T18:27:21.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:27:21.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.323+0000 7f690c4dc640 1 -- 172.21.15.37:0/4109884436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 msgr2=0x7f690410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.323+0000 7f690c4dc640 1 --2- 172.21.15.37:0/4109884436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f690410dd60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009ca0 tx=0x7f68fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 -- 172.21.15.37:0/4109884436 shutdown_connections 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 --2- 172.21.15.37:0/4109884436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f690410dd60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 --2- 172.21.15.37:0/4109884436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6904101ac0 0x7f6904105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 --2- 172.21.15.37:0/4109884436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6904101110 0x7f69041014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 -- 172.21.15.37:0/4109884436 >> 172.21.15.37:0/4109884436 conn(0x7f6904077fb0 msgr2=0x7f69040783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 -- 172.21.15.37:0/4109884436 shutdown_connections 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 -- 172.21.15.37:0/4109884436 wait complete. 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 Processor -- start 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 -- start start 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6904101110 0x7f6904198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.324+0000 7f690c4dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f6904198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690c4dc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f690419cfc0 0x7f690419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690c4dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6904112240 con 0x7f6904106190 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690c4dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69041120c0 con 0x7f6904101110 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690c4dc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69041123c0 con 0x7f690419cfc0 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f6904198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f6904198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60512/0 (socket says 172.21.15.37:60512) 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 -- 172.21.15.37:0/2392409709 learned_addr learned my addr 172.21.15.37:0/2392409709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690aa79640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f690419cfc0 0x7f690419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690a278640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6904101110 0x7f6904198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 -- 172.21.15.37:0/2392409709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f690419cfc0 msgr2=0x7f690419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f690419cfc0 0x7f690419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 -- 172.21.15.37:0/2392409709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6904101110 msgr2=0x7f6904198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6904101110 0x7f6904198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 -- 172.21.15.37:0/2392409709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690419dab0 con 0x7f6904106190 2024-04-23T18:27:21.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f6909a77640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f6904198c40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4017070 con 0x7f6904106190 2024-04-23T18:27:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690419dda0 con 0x7f6904106190 2024-04-23T18:27:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f4009c30 con 0x7f6904106190 2024-04-23T18:27:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003d30 con 0x7f6904106190 2024-04-23T18:27:21.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.325+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a5590 con 0x7f6904106190 2024-04-23T18:27:21.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.327+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69040691c0 con 0x7f6904106190 2024-04-23T18:27:21.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.328+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68f4003ed0 con 0x7f6904106190 2024-04-23T18:27:21.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.328+0000 7f68fb7fe640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68f00a3d20 0x7f68f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.328+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f68f40d0340 con 0x7f6904106190 2024-04-23T18:27:21.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.328+0000 7f690a278640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68f00a3d20 0x7f68f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.329+0000 7f690a278640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68f00a3d20 0x7f68f00a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6900005fd0 tx=0x7f690000b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.333+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68f4020030 con 0x7f6904106190 2024-04-23T18:27:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.559+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6904101ac0 con 0x7f6904106190 2024-04-23T18:27:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.560+0000 7f68fb7fe640 1 -- 172.21.15.37:0/2392409709 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11881 (secure 0 0 0) 0x7f68f409c9d0 con 0x7f6904106190 2024-04-23T18:27:21.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:27:21.553 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:26:42.565223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:27:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 34 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68f00a3d20 msgr2=0x7f68f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68f00a3d20 0x7f68f00a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6900005fd0 tx=0x7f690000b390 comp rx=0 tx=0).stop 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 msgr2=0x7f6904198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f6904198c40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f400dde0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 shutdown_connections 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68f00a3d20 0x7f68f00a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f690419cfc0 0x7f690419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6904106190 0x7f6904198c40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 --2- 172.21.15.37:0/2392409709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6904101110 0x7f6904198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 >> 172.21.15.37:0/2392409709 conn(0x7f6904077fb0 msgr2=0x7f69040fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 shutdown_connections 2024-04-23T18:27:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.562+0000 7f690c4dc640 1 -- 172.21.15.37:0/2392409709 wait complete. 2024-04-23T18:27:21.566 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (h) to a remote file /tmp/mds.h-ops-9f848727-1 2024-04-23T18:27:21.566 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 ops --flags=locks --path=/tmp/mds.h-ops-9f848727-1 2024-04-23T18:27:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 -- 172.21.15.37:0/345429480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c10c570 msgr2=0x7ff22c10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 --2- 172.21.15.37:0/345429480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c10c570 0x7ff22c10e9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff224009ca0 tx=0x7ff22402a950 comp rx=0 tx=0).stop 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 -- 172.21.15.37:0/345429480 shutdown_connections 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 --2- 172.21.15.37:0/345429480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c10c570 0x7ff22c10e9c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 --2- 172.21.15.37:0/345429480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c103970 0x7ff22c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 --2- 172.21.15.37:0/345429480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff22c103050 0x7ff22c103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 -- 172.21.15.37:0/345429480 >> 172.21.15.37:0/345429480 conn(0x7ff22c077fb0 msgr2=0x7ff22c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 -- 172.21.15.37:0/345429480 shutdown_connections 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 -- 172.21.15.37:0/345429480 wait complete. 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.680+0000 7ff23366b640 1 Processor -- start 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 -- start start 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 0x7ff22c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c103970 0x7ff22c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff22c10c570 0x7ff22c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1121a0 con 0x7ff22c103050 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c112020 con 0x7ff22c10c570 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff23366b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c112320 con 0x7ff22c103970 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 0x7ff22c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 0x7ff22c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60534/0 (socket says 172.21.15.37:60534) 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 -- 172.21.15.37:0/3119541674 learned_addr learned my addr 172.21.15.37:0/3119541674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff230c06640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c103970 0x7ff22c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231c08640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff22c10c570 0x7ff22c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 -- 172.21.15.37:0/3119541674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c103970 msgr2=0x7ff22c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c103970 0x7ff22c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 -- 172.21.15.37:0/3119541674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff22c10c570 msgr2=0x7ff22c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff22c10c570 0x7ff22c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 -- 172.21.15.37:0/3119541674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c19d730 con 0x7ff22c103050 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff231407640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 0x7ff22c198740 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff22800d950 tx=0x7ff22800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.681+0000 7ff2227fc640 1 -- 172.21.15.37:0/3119541674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228017070 con 0x7ff22c103050 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.682+0000 7ff2227fc640 1 -- 172.21.15.37:0/3119541674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff228009c30 con 0x7ff22c103050 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.682+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c19da20 con 0x7ff22c103050 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.682+0000 7ff2227fc640 1 -- 172.21.15.37:0/3119541674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22800f620 con 0x7ff22c103050 2024-04-23T18:27:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.682+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1a52b0 con 0x7ff22c103050 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff2227fc640 1 -- 172.21.15.37:0/3119541674 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff228010460 con 0x7ff22c103050 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 Processor -- start 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 -- start start 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df01d20 0x56482df02100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56482de672e0 con 0x56482df02640 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56482de04060 con 0x56482df01d20 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff23637c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56482de632a0 con 0x56482df0cd10 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60542/0 (socket says 172.21.15.37:60542) 2024-04-23T18:27:21.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.684+0000 7ff231407640 1 -- 172.21.15.37:0/739626395 learned_addr learned my addr 172.21.15.37:0/739626395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff2227fc640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff2180a3da0 0x7ff2180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff230c06640 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231407640 1 -- 172.21.15.37:0/739626395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 msgr2=0x56482df0f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231407640 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df0f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231407640 1 -- 172.21.15.37:0/739626395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df01d20 msgr2=0x56482df02100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231c08640 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df01d20 0x56482df02100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231407640 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df01d20 0x56482df02100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231407640 1 -- 172.21.15.37:0/739626395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482de2cd90 con 0x56482df02640 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231c08640 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df01d20 0x56482df02100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231c08640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff2180a3da0 0x7ff2180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff2227fc640 1 -- 172.21.15.37:0/3119541674 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff2280d03f0 con 0x7ff22c103050 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff231407640 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0c7d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff2280cdad0 tx=0x7ff2280209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace97d69f6ae3d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff2177fe640 1 -- 172.21.15.37:0/739626395 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22809bec0 con 0x56482df02640 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff2177fe640 1 -- 172.21.15.37:0/739626395 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22809ca80 con 0x56482df02640 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.685+0000 7ff2177fe640 1 -- 172.21.15.37:0/739626395 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22809bce0 con 0x56482df02640 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 -- 172.21.15.37:0/739626395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 msgr2=0x56482df0c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff231c08640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff2180a3da0 0x7ff2180a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff224009a60 tx=0x7ff224009890 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0c7d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff2280cdad0 tx=0x7ff2280209d0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 -- 172.21.15.37:0/739626395 shutdown_connections 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df0f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0c7d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 --2- 172.21.15.37:0/739626395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df01d20 0x56482df02100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 -- 172.21.15.37:0/739626395 >> 172.21.15.37:0/739626395 conn(0x56482defe040 msgr2=0x56482defe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 -- 172.21.15.37:0/739626395 shutdown_connections 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 -- 172.21.15.37:0/739626395 wait complete. 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 -- start start 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.686+0000 7ff23637c740 1 Processor -- start 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df09c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df02640 0x56482df0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 0x56482df0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56482df0fb40 con 0x56482df01d20 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56482de019e0 con 0x56482df0cd10 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56482df0fcc0 con 0x56482df02640 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df09c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff23637c740 10 client.0 _ll_register_callbacks cb 0x56482de15230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:21.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df09c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60544/0 (socket says 172.21.15.37:60544) 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 -- 172.21.15.37:0/1497822641 learned_addr learned my addr 172.21.15.37:0/1497822641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff230c06640 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df02640 0x56482df0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 -- 172.21.15.37:0/1497822641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df02640 msgr2=0x56482df0a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df02640 0x56482df0a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 -- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 msgr2=0x56482df0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 0x56482df0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 -- 172.21.15.37:0/1497822641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482db79730 con 0x56482df01d20 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff231407640 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df09c90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2280987e0 tx=0x7ff22800dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464715a930a30c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.687+0000 7ff214ff9640 1 -- 172.21.15.37:0/1497822641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22809bce0 con 0x56482df01d20 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff214ff9640 1 -- 172.21.15.37:0/1497822641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff2280cddb0 con 0x56482df01d20 2024-04-23T18:27:21.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22809ca80 con 0x56482df01d20 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff214ff9640 1 -- 172.21.15.37:0/1497822641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff22809bec0 con 0x56482df01d20 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 10 client.16365 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56482dc45b40 con 0x56482df01d20 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff214ff9640 1 -- 172.21.15.37:0/1497822641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22809ca80 con 0x56482df01d20 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff214ff9640 1 -- 172.21.15.37:0/1497822641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff2280989f0 con 0x56482df01d20 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 10 client.16365 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 10 client.16365 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 20 client.16365 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df1cee0 0x56482df1f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 4 client.16365 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.688+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56482de74d00 con 0x56482df1cee0 2024-04-23T18:27:21.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.689+0000 7ff230c06640 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df1cee0 0x56482df1f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.689+0000 7ff230c06640 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df1cee0 0x56482df1f2d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cff1fdacf0c640cc server_cookie=b27cca40f3947e10 in_seq=0 out_seq=0 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.689+0000 7ff214ff9640 10 client.16365 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff214ff9640 1 -- 172.21.15.37:0/1497822641 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56482de74d00 con 0x56482df1cee0 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff214ff9640 10 client.16365 handle_command_reply: tid=0 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 client.16365 shutdown 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 msgr2=0x56482df09c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df09c90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2280987e0 tx=0x7ff22800dfa0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 shutdown_connections 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df1cee0 0x56482df1f2d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 0x56482df0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df02640 0x56482df0a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff214ff9640 0 client.16365 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 --2- 172.21.15.37:0/1497822641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df09c90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 >> 172.21.15.37:0/1497822641 conn(0x56482defe040 msgr2=0x56482df03d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 shutdown_connections 2024-04-23T18:27:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.690+0000 7ff23637c740 1 -- 172.21.15.37:0/1497822641 wait complete. 2024-04-23T18:27:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.691+0000 7ff23637c740 20 client.16365 trim_cache size 0 max 16384 2024-04-23T18:27:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.706+0000 7ff23637c740 1 Processor -- start 2024-04-23T18:27:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.706+0000 7ff23637c740 1 -- start start 2024-04-23T18:27:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff23637c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df01d20 0x56482df18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff23637c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff23637c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 0x56482df248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff23637c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1b05c0 con 0x56482df02640 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff23637c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ade30 con 0x56482df0cd10 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff23637c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1abc80 con 0x56482df01d20 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60548/0 (socket says 172.21.15.37:60548) 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 -- 172.21.15.37:0/1428819820 learned_addr learned my addr 172.21.15.37:0/1428819820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff231407640 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df01d20 0x56482df18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 -- 172.21.15.37:0/1428819820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df01d20 msgr2=0x56482df18a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df01d20 0x56482df18a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 -- 172.21.15.37:0/1428819820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 msgr2=0x56482df248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 0x56482df248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 -- 172.21.15.37:0/1428819820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482df081e0 con 0x56482df02640 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff231407640 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df01d20 0x56482df18a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff230c06640 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0a530 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009a50 tx=0x7ff21c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecbd94f3722ede9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff216ffd640 1 -- 172.21.15.37:0/1428819820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c003a40 con 0x56482df02640 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff216ffd640 1 -- 172.21.15.37:0/1428819820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff21c01e8d0 con 0x56482df02640 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.707+0000 7ff216ffd640 1 -- 172.21.15.37:0/1428819820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0047a0 con 0x56482df02640 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 -- 172.21.15.37:0/1428819820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 msgr2=0x56482df0a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0a530 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009a50 tx=0x7ff21c004410 comp rx=0 tx=0).stop 2024-04-23T18:27:21.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 -- 172.21.15.37:0/1428819820 shutdown_connections 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df0cd10 0x56482df248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df02640 0x56482df0a530 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 --2- 172.21.15.37:0/1428819820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df01d20 0x56482df18a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 -- 172.21.15.37:0/1428819820 >> 172.21.15.37:0/1428819820 conn(0x56482defe040 msgr2=0x56482defd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 -- 172.21.15.37:0/1428819820 shutdown_connections 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 -- 172.21.15.37:0/1428819820 wait complete. 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 -- start start 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.708+0000 7ff23637c740 1 Processor -- start 2024-04-23T18:27:21.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df02640 0x56482df11620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ac2f0 con 0x56482df01d20 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ad760 con 0x56482df02640 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ac9c0 con 0x56482df0cd10 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 10 client.0 _ll_register_callbacks cb 0x7ff2280984e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60554/0 (socket says 172.21.15.37:60554) 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 -- 172.21.15.37:0/3430211506 learned_addr learned my addr 172.21.15.37:0/3430211506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff230c06640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df02640 0x56482df11620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231c08640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 -- 172.21.15.37:0/3430211506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 msgr2=0x56482df82140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df82140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 -- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df02640 msgr2=0x56482df11620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df02640 0x56482df11620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 -- 172.21.15.37:0/3430211506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482df826d0 con 0x56482df01d20 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff230c06640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df02640 0x56482df11620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231c08640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff231407640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df110e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff2280a5160 tx=0x7ff22800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf705dad16a98f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff214ff9640 1 -- 172.21.15.37:0/3430211506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2280b4050 con 0x56482df01d20 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff214ff9640 1 -- 172.21.15.37:0/3430211506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff22809b7d0 con 0x56482df01d20 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21c01e8d0 con 0x56482df01d20 2024-04-23T18:27:21.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff214ff9640 1 -- 172.21.15.37:0/3430211506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff22809bce0 con 0x56482df01d20 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.709+0000 7ff23637c740 10 client.16371 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56482df839b0 con 0x56482df01d20 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff214ff9640 1 -- 172.21.15.37:0/3430211506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff21c01e8d0 con 0x56482df01d20 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff214ff9640 1 -- 172.21.15.37:0/3430211506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff22809b240 con 0x56482df01d20 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 10 client.16371 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 10 client.16371 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 20 client.16371 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df83fd0 0x56482df863f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 4 client.16371 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.h-ops-9f848727-1"}] 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 --> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.h-ops-9f848727-1"}) -- 0x56482de6eb70 con 0x56482df83fd0 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.710+0000 7ff230c06640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df83fd0 0x56482df863f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.711+0000 7ff214ff9640 10 client.16371 ms_handle_connect on v2:172.21.15.172:6836/364708999 2024-04-23T18:27:21.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.711+0000 7ff230c06640 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df83fd0 0x56482df863f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8320bc35a7c81a03 server_cookie=21de87a029baba4a in_seq=0 out_seq=0 2024-04-23T18:27:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff214ff9640 1 -- 172.21.15.37:0/3430211506 <== mds.2 v2:172.21.15.172:6836/364708999 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56482de6eb70 con 0x56482df83fd0 2024-04-23T18:27:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff214ff9640 10 client.16371 handle_command_reply: tid=0 2024-04-23T18:27:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 client.16371 shutdown 2024-04-23T18:27:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 msgr2=0x56482df110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df110e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff2280a5160 tx=0x7ff22800bf80 comp rx=0 tx=0).stop 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 shutdown_connections 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:6836/364708999,v1:172.21.15.172:6837/364708999] conn(0x56482df83fd0 0x56482df863f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56482df0cd10 0x56482df82140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56482df02640 0x56482df11620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff214ff9640 0 client.16371 ms_handle_reset on v2:172.21.15.172:6836/364708999 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 --2- 172.21.15.37:0/3430211506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56482df01d20 0x56482df110e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.716+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 >> 172.21.15.37:0/3430211506 conn(0x56482defe040 msgr2=0x56482df0bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 shutdown_connections 2024-04-23T18:27:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23637c740 1 -- 172.21.15.37:0/3430211506 wait complete. 2024-04-23T18:27:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23637c740 20 client.16371 trim_cache size 0 max 16384 2024-04-23T18:27:21.713 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:27:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/tmp/mds.h-ops-9f848727-1" 2024-04-23T18:27:21.713 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:27:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff2180a3da0 msgr2=0x7ff2180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff2180a3da0 0x7ff2180a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff224009a60 tx=0x7ff224009890 comp rx=0 tx=0).stop 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 msgr2=0x7ff22c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 0x7ff22c198740 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff22800d950 tx=0x7ff22800de10 comp rx=0 tx=0).stop 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 shutdown_connections 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff2180a3da0 0x7ff2180a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff22c10c570 0x7ff22c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c103970 0x7ff22c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 --2- 172.21.15.37:0/3119541674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c103050 0x7ff22c198740 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 >> 172.21.15.37:0/3119541674 conn(0x7ff22c077fb0 msgr2=0x7ff22c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 shutdown_connections 2024-04-23T18:27:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.717+0000 7ff23366b640 1 -- 172.21.15.37:0/3119541674 wait complete. 2024-04-23T18:27:21.720 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (i) to a remote file /tmp/mds.i-ops-9f848727-1 2024-04-23T18:27:21.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 ops --flags=locks --path=/tmp/mds.i-ops-9f848727-1 2024-04-23T18:27:21.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/2522865687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 msgr2=0x7f8eb810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/2522865687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb810dc80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009cd0 tx=0x7f8eb002a8e0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/2522865687 shutdown_connections 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/2522865687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb810dc80 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/2522865687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 0x7f8eb8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/2522865687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eb8107030 0x7f8eb8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/2522865687 >> 172.21.15.37:0/2522865687 conn(0x7f8eb8076c50 msgr2=0x7f8eb8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/2522865687 shutdown_connections 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/2522865687 wait complete. 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 Processor -- start 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.834+0000 7f8ebd3f7640 1 -- start start 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 0x7f8eb8102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eb8107030 0x7f8eb81035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81110e0 con 0x7f8eb8101b40 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8110f60 con 0x7f8eb81011a0 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111260 con 0x7f8eb8107030 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb8103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60572/0 (socket says 172.21.15.37:60572) 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 -- 172.21.15.37:0/4179752698 learned_addr learned my addr 172.21.15.37:0/4179752698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eb8107030 0x7f8eb81035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 0x7f8eb8102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 -- 172.21.15.37:0/4179752698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eb8107030 msgr2=0x7f8eb81035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eb8107030 0x7f8eb81035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 -- 172.21.15.37:0/4179752698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 msgr2=0x7f8eb8102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 0x7f8eb8102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 -- 172.21.15.37:0/4179752698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb81a6c70 con 0x7f8eb8101b40 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 0x7f8eb8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb8103090 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00d950 tx=0x7f8eac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8e9ffff640 1 -- 172.21.15.37:0/4179752698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac017070 con 0x7f8eb8101b40 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb81a6f60 con 0x7f8eb8101b40 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8e9ffff640 1 -- 172.21.15.37:0/4179752698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac009c30 con 0x7f8eb8101b40 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.835+0000 7f8e9ffff640 1 -- 172.21.15.37:0/4179752698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00f620 con 0x7f8eb8101b40 2024-04-23T18:27:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.836+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a7420 con 0x7f8eb8101b40 2024-04-23T18:27:21.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.42 MB/sec execute 206 sec latency 43038.579 ms 2024-04-23T18:27:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8e9ffff640 1 -- 172.21.15.37:0/4179752698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8eac010460 con 0x7f8eb8101b40 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 Processor -- start 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 -- start start 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703edd20 0x5585703ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 0x5585703fb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585703532e0 con 0x5585703f8d10 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5585702f0060 con 0x5585703ee640 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55857034f2a0 con 0x5585703edd20 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8e9ffff640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8ea00a3d20 0x7f8ea00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.838+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 0x5585703fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 0x5585703fb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60584/0 (socket says 172.21.15.37:60584) 2024-04-23T18:27:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 -- 172.21.15.37:0/3051567048 learned_addr learned my addr 172.21.15.37:0/3051567048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703edd20 0x5585703ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8ea00a3d20 0x7f8ea00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8e9ffff640 1 -- 172.21.15.37:0/4179752698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8eac0d0540 con 0x7f8eb8101b40 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 -- 172.21.15.37:0/3051567048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703edd20 msgr2=0x5585703ee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703edd20 0x5585703ee100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 -- 172.21.15.37:0/3051567048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 msgr2=0x5585703f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 -- 172.21.15.37:0/3051567048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558570318d90 con 0x5585703f8d10 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 0x5585703fb100 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8eac09c630 tx=0x7f8eac00dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d932a753b447465 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8ea00a3d20 0x7f8ea00a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0033040 tx=0x7f8eb0005f30 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8e9d7fa640 1 -- 172.21.15.37:0/3051567048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac020070 con 0x5585703f8d10 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8e9d7fa640 1 -- 172.21.15.37:0/3051567048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac009de0 con 0x5585703f8d10 2024-04-23T18:27:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.839+0000 7f8e9d7fa640 1 -- 172.21.15.37:0/3051567048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0d3b60 con 0x5585703f8d10 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 -- 172.21.15.37:0/3051567048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 msgr2=0x5585703fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 0x5585703fb100 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8eac09c630 tx=0x7f8eac00dfa0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 -- 172.21.15.37:0/3051567048 shutdown_connections 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703f8d10 0x5585703fb100 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/3051567048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703edd20 0x5585703ee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 -- 172.21.15.37:0/3051567048 >> 172.21.15.37:0/3051567048 conn(0x5585703ea040 msgr2=0x5585703ea420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 -- 172.21.15.37:0/3051567048 shutdown_connections 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 -- 172.21.15.37:0/3051567048 wait complete. 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 -- start start 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.840+0000 7f8ec01e7740 1 Processor -- start 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703ee640 0x5585703f6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703f8d10 0x5585703f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585703fbae0 con 0x5585703edd20 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5585702ed9e0 con 0x5585703f8d10 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5585703fbc60 con 0x5585703ee640 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 10 client.0 _ll_register_callbacks cb 0x558570301230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60600/0 (socket says 172.21.15.37:60600) 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/2560075136 learned_addr learned my addr 172.21.15.37:0/2560075136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703ee640 0x5585703f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703f8d10 0x5585703f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703ee640 msgr2=0x5585703f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703ee640 0x5585703f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/2560075136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703f8d10 msgr2=0x5585703f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703f8d10 0x5585703f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/2560075136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558570065730 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703f8d10 0x5585703f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f5c50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8010780 tx=0x7f8ea8010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedeb9e2058321e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8e9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8e9aff5640 1 -- 172.21.15.37:0/2560075136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801d060 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8e9aff5640 1 -- 172.21.15.37:0/2560075136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8ea800eda0 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.841+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eac009de0 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8e9aff5640 1 -- 172.21.15.37:0/2560075136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8018430 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 10 client.16383 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558570131b40 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8e9aff5640 1 -- 172.21.15.37:0/2560075136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac009de0 con 0x5585703edd20 2024-04-23T18:27:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8e9aff5640 1 -- 172.21.15.37:0/2560075136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8ea800d4c0 con 0x5585703edd20 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 10 client.16383 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 10 client.16383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 20 client.16383 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570409100 0x55857040b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 4 client.16383 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558570360d00 con 0x558570409100 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.842+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570409100 0x55857040b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.843+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570409100 0x55857040b550 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1d4ecee3cce0094 server_cookie=ee0f3d8a1ce92e37 in_seq=0 out_seq=0 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.843+0000 7f8e9aff5640 10 client.16383 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.843+0000 7f8e9aff5640 1 -- 172.21.15.37:0/2560075136 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558570360d00 con 0x558570409100 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.843+0000 7f8e9aff5640 10 client.16383 handle_command_reply: tid=0 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.843+0000 7f8ec01e7740 1 client.16383 shutdown 2024-04-23T18:27:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 msgr2=0x5585703f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f5c50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8010780 tx=0x7f8ea8010c40 comp rx=0 tx=0).stop 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 shutdown_connections 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570409100 0x55857040b550 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703f8d10 0x5585703f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703ee640 0x5585703f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/2560075136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f5c50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 >> 172.21.15.37:0/2560075136 conn(0x5585703ea040 msgr2=0x5585703efe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8e9aff5640 0 client.16383 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 shutdown_connections 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 1 -- 172.21.15.37:0/2560075136 wait complete. 2024-04-23T18:27:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.844+0000 7f8ec01e7740 20 client.16383 trim_cache size 0 max 16384 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 Processor -- start 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 -- start start 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703edd20 0x5585704049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 0x5585703f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x558570410e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b27a0 con 0x5585703ee640 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b0010 con 0x5585703edd20 2024-04-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ade60 con 0x5585703f8d10 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 0x5585703f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 0x5585703f64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60614/0 (socket says 172.21.15.37:60614) 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 -- 172.21.15.37:0/960229221 learned_addr learned my addr 172.21.15.37:0/960229221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703edd20 0x5585704049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 -- 172.21.15.37:0/960229221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 msgr2=0x558570410e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x558570410e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x558570410e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 -- 172.21.15.37:0/960229221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703edd20 msgr2=0x5585704049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703edd20 0x5585704049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 -- 172.21.15.37:0/960229221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585703ee470 con 0x5585703ee640 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703edd20 0x5585704049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.860+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 0x5585703f64f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8eac098e40 tx=0x7f8eac098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a148250709cffba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8e9cff9640 1 -- 172.21.15.37:0/960229221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0b5070 con 0x5585703ee640 2024-04-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8e9cff9640 1 -- 172.21.15.37:0/960229221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac0a0820 con 0x5585703ee640 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8e9cff9640 1 -- 172.21.15.37:0/960229221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac020040 con 0x5585703ee640 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 -- 172.21.15.37:0/960229221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 msgr2=0x5585703f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 0x5585703f64f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8eac098e40 tx=0x7f8eac098040 comp rx=0 tx=0).stop 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 -- 172.21.15.37:0/960229221 shutdown_connections 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x558570410e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703ee640 0x5585703f64f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/960229221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703edd20 0x5585704049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 -- 172.21.15.37:0/960229221 >> 172.21.15.37:0/960229221 conn(0x5585703ea040 msgr2=0x5585703f7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 -- 172.21.15.37:0/960229221 shutdown_connections 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 -- 172.21.15.37:0/960229221 wait complete. 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.861+0000 7f8ec01e7740 1 -- start start 2024-04-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 Processor -- start 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x55857046e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ae4d0 con 0x5585703edd20 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81af940 con 0x5585703ee640 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81aeba0 con 0x5585703f8d10 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8ec01e7740 10 client.0 _ll_register_callbacks cb 0x7f8ea800b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60630/0 (socket says 172.21.15.37:60630) 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/184234237 learned_addr learned my addr 172.21.15.37:0/184234237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb77fe640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x55857046e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 msgr2=0x55857046e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x55857046e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/184234237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 msgr2=0x5585703f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 -- 172.21.15.37:0/184234237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55857046ec20 con 0x5585703edd20 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8eb6ffd640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f3070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800ee50 tx=0x7f8ea800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fd2dac81903688 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8e9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.862+0000 7f8e9aff5640 1 -- 172.21.15.37:0/184234237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800e780 con 0x5585703edd20 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8e9aff5640 1 -- 172.21.15.37:0/184234237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8ea8010020 con 0x5585703edd20 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eac0a0820 con 0x5585703edd20 2024-04-23T18:27:21.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8e9aff5640 1 -- 172.21.15.37:0/184234237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ea801d030 con 0x5585703edd20 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 10 client.16389 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55857046ff40 con 0x5585703edd20 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8e9aff5640 1 -- 172.21.15.37:0/184234237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac0a0820 con 0x5585703edd20 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8e9aff5640 1 -- 172.21.15.37:0/184234237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8ea80047c0 con 0x5585703edd20 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 10 client.16389 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 10 client.16389 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 20 client.16389 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570470740 0x558570472b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 4 client.16389 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.i-ops-9f848727-1"}] 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 --> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.i-ops-9f848727-1"}) -- 0x55857035ab70 con 0x558570470740 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.863+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570470740 0x558570472b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.864+0000 7f8e9aff5640 10 client.16389 ms_handle_connect on v2:172.21.15.173:6838/496716250 2024-04-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.864+0000 7f8eb67fc640 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570470740 0x558570472b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3c55505118f7bbf server_cookie=8dbd220e411b5efa in_seq=0 out_seq=0 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8e9aff5640 1 -- 172.21.15.37:0/184234237 <== mds.1 v2:172.21.15.173:6838/496716250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55857035ab70 con 0x558570470740 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8e9aff5640 10 client.16389 handle_command_reply: tid=0 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 client.16389 shutdown 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 msgr2=0x5585703f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f3070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800ee50 tx=0x7f8ea800ee80 comp rx=0 tx=0).stop 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 shutdown_connections 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:6838/496716250,v1:172.21.15.173:6839/496716250] conn(0x558570470740 0x558570472b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5585703f8d10 0x55857046e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8e9aff5640 0 client.16389 ms_handle_reset on v2:172.21.15.173:6838/496716250 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585703ee640 0x5585703f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 --2- 172.21.15.37:0/184234237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585703edd20 0x5585703f3070 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 >> 172.21.15.37:0/184234237 conn(0x5585703ea040 msgr2=0x5585703f7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.867+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 shutdown_connections 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ec01e7740 1 -- 172.21.15.37:0/184234237 wait complete. 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ec01e7740 20 client.16389 trim_cache size 0 max 16384 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/tmp/mds.i-ops-9f848727-1" 2024-04-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8ea00a3d20 msgr2=0x7f8ea00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8ea00a3d20 0x7f8ea00a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0033040 tx=0x7f8eb0005f30 comp rx=0 tx=0).stop 2024-04-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 msgr2=0x7f8eb8103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb8103090 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00d950 tx=0x7f8eac00de10 comp rx=0 tx=0).stop 2024-04-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 shutdown_connections 2024-04-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f8ea00a3d20 0x7f8ea00a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eb8107030 0x7f8eb81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb8101b40 0x7f8eb8103090 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 --2- 172.21.15.37:0/4179752698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb81011a0 0x7f8eb8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 >> 172.21.15.37:0/4179752698 conn(0x7f8eb8076c50 msgr2=0x7f8eb806a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 shutdown_connections 2024-04-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.868+0000 7f8ebd3f7640 1 -- 172.21.15.37:0/4179752698 wait complete. 2024-04-23T18:27:21.871 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /tmp/mds.b-ops-9f848727-1 2024-04-23T18:27:21.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 ops --flags=locks --path=/tmp/mds.b-ops-9f848727-1 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 -- 172.21.15.37:0/2899346374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1014a0 msgr2=0x7fba7c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 --2- 172.21.15.37:0/2899346374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1014a0 0x7fba7c106bd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fba70009900 tx=0x7fba7002a410 comp rx=0 tx=0).stop 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 -- 172.21.15.37:0/2899346374 shutdown_connections 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 --2- 172.21.15.37:0/2899346374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c1072b0 0x7fba7c10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 --2- 172.21.15.37:0/2899346374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1014a0 0x7fba7c106bd0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 --2- 172.21.15.37:0/2899346374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c100af0 0x7fba7c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 -- 172.21.15.37:0/2899346374 >> 172.21.15.37:0/2899346374 conn(0x7fba7c0760e0 msgr2=0x7fba7c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 -- 172.21.15.37:0/2899346374 shutdown_connections 2024-04-23T18:27:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.984+0000 7fba82907640 1 -- 172.21.15.37:0/2899346374 wait complete. 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 Processor -- start 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 -- start start 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c100af0 0x7fba7c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 0x7fba7c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c198590 0x7fba7c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c112230 con 0x7fba7c1072b0 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1120b0 con 0x7fba7c198590 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba82907640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1123b0 con 0x7fba7c100af0 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 0x7fba7c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 0x7fba7c19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60650/0 (socket says 172.21.15.37:60650) 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 -- 172.21.15.37:0/3227231554 learned_addr learned my addr 172.21.15.37:0/3227231554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7bfff640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c100af0 0x7fba7c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba80ea4640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c198590 0x7fba7c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 -- 172.21.15.37:0/3227231554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c100af0 msgr2=0x7fba7c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c100af0 0x7fba7c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 -- 172.21.15.37:0/3227231554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c198590 msgr2=0x7fba7c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c198590 0x7fba7c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7b7fe640 1 -- 172.21.15.37:0/3227231554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1990b0 con 0x7fba7c1072b0 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba7bfff640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c100af0 0x7fba7c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.985+0000 7fba80ea4640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c198590 0x7fba7c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.986+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 0x7fba7c19e920 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fba70032fd0 tx=0x7fba70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.986+0000 7fba797fa640 1 -- 172.21.15.37:0/3227231554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7003f030 con 0x7fba7c1072b0 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.986+0000 7fba797fa640 1 -- 172.21.15.37:0/3227231554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba70004490 con 0x7fba7c1072b0 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.986+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c199340 con 0x7fba7c1072b0 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.986+0000 7fba797fa640 1 -- 172.21.15.37:0/3227231554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70035070 con 0x7fba7c1072b0 2024-04-23T18:27:21.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.986+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1a5530 con 0x7fba7c1072b0 2024-04-23T18:27:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.988+0000 7fba797fa640 1 -- 172.21.15.37:0/3227231554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba70042030 con 0x7fba7c1072b0 2024-04-23T18:27:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.988+0000 7fba8561a740 1 Processor -- start 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.988+0000 7fba8561a740 1 -- start start 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba8561a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa97fc20 0x55bcaa980000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba8561a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa98ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba8561a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa991510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba8561a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcaa8e52e0 con 0x55bcaa980540 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba8561a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bcaa882060 con 0x55bcaa98f120 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba8561a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bcaa8e12a0 con 0x55bcaa97fc20 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa98ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa98ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60656/0 (socket says 172.21.15.37:60656) 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 -- 172.21.15.37:0/2389861172 learned_addr learned my addr 172.21.15.37:0/2389861172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba797fa640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fba640a3d20 0x7fba640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa991510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba80ea4640 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa97fc20 0x55bcaa980000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba80ea4640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fba640a3d20 0x7fba640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 -- 172.21.15.37:0/2389861172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa97fc20 msgr2=0x55bcaa980000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa97fc20 0x55bcaa980000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 -- 172.21.15.37:0/2389861172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 msgr2=0x55bcaa991510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa991510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 -- 172.21.15.37:0/2389861172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcaa8aad90 con 0x55bcaa980540 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa991510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba797fa640 1 -- 172.21.15.37:0/3227231554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fba700f0dc0 con 0x7fba7c1072b0 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba7bfff640 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa98ebe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00bcc0 tx=0x7fba6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc0feeef03f66e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba80ea4640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fba640a3d20 0x7fba640a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba74005c40 tx=0x7fba74009200 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba627fc640 1 -- 172.21.15.37:0/2389861172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005020 con 0x55bcaa980540 2024-04-23T18:27:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba627fc640 1 -- 172.21.15.37:0/2389861172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c0047a0 con 0x55bcaa980540 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.989+0000 7fba627fc640 1 -- 172.21.15.37:0/2389861172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0116a0 con 0x55bcaa980540 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 -- 172.21.15.37:0/2389861172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 msgr2=0x55bcaa98ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa98ebe0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00bcc0 tx=0x7fba6c00f730 comp rx=0 tx=0).stop 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 -- 172.21.15.37:0/2389861172 shutdown_connections 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa991510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa98ebe0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 --2- 172.21.15.37:0/2389861172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa97fc20 0x55bcaa980000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 -- 172.21.15.37:0/2389861172 >> 172.21.15.37:0/2389861172 conn(0x55bcaa97c040 msgr2=0x55bcaa97c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 -- 172.21.15.37:0/2389861172 shutdown_connections 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 -- 172.21.15.37:0/2389861172 wait complete. 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.990+0000 7fba8561a740 1 -- start start 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 Processor -- start 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa98c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa980540 0x55bcaa98c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa98cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcaa991f80 con 0x55bcaa97fc20 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bcaa87f9e0 con 0x55bcaa980540 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bcaa992100 con 0x55bcaa98f120 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba8561a740 10 client.0 _ll_register_callbacks cb 0x55bcaa893230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa98c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa98c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60660/0 (socket says 172.21.15.37:60660) 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 -- 172.21.15.37:0/2870815819 learned_addr learned my addr 172.21.15.37:0/2870815819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba80ea4640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa98cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa980540 0x55bcaa98c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 -- 172.21.15.37:0/2870815819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 msgr2=0x55bcaa98cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa98cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 -- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa980540 msgr2=0x55bcaa98c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa980540 0x55bcaa98c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:21.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7bfff640 1 -- 172.21.15.37:0/2870815819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcaa5f7730 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.991+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa980540 0x55bcaa98c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba7bfff640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa98c090 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527dbec0b0f7005b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba5b7fe640 1 -- 172.21.15.37:0/2870815819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c02a060 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba5b7fe640 1 -- 172.21.15.37:0/2870815819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fba6c005000 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6c0047a0 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba5b7fe640 1 -- 172.21.15.37:0/2870815819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba6c005200 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 10 client.16401 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bcaa6c3b40 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba5b7fe640 1 -- 172.21.15.37:0/2870815819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c0047a0 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba5b7fe640 1 -- 172.21.15.37:0/2870815819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fba6c0037b0 con 0x55bcaa97fc20 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 10 client.16401 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:21.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 20 client.16401 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaa99f4d0 0x55bcaa9a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 4 client.16401 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.992+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcaa8f2d00 con 0x55bcaa99f4d0 2024-04-23T18:27:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.993+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaa99f4d0 0x55bcaa9a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.993+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaa99f4d0 0x55bcaa9a18c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322038dcfa6b444 server_cookie=63184745917ed87b in_seq=0 out_seq=0 2024-04-23T18:27:21.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:21.993+0000 7fba5b7fe640 10 client.16401 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:22.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:22 smithi172 ceph-mon[32456]: pgmap v211: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-23T18:27:22.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:22 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2392409709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:22.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:22 smithi173 ceph-mon[32906]: pgmap v211: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-23T18:27:22.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:22 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2392409709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:22.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:22 smithi037 ceph-mon[29404]: pgmap v211: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-23T18:27:22.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:22 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2392409709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:22.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.41 MB/sec execute 207 sec latency 44038.612 ms 2024-04-23T18:27:23.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.39 MB/sec execute 208 sec latency 45038.649 ms 2024-04-23T18:27:24.407 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:24 smithi173 ceph-mon[32906]: pgmap v212: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-23T18:27:24.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:24 smithi172 ceph-mon[32456]: pgmap v212: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-23T18:27:24.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:24 smithi037 ceph-mon[29404]: pgmap v212: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-04-23T18:27:24.829 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.38 MB/sec execute 209 sec latency 46038.689 ms 2024-04-23T18:27:25.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.36 MB/sec execute 210 sec latency 47038.802 ms 2024-04-23T18:27:26.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:26 smithi172 ceph-mon[32456]: pgmap v213: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-23T18:27:26.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:26 smithi173 ceph-mon[32906]: pgmap v213: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-23T18:27:26.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:26 smithi037 ceph-mon[29404]: pgmap v213: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-23T18:27:26.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.34 MB/sec execute 211 sec latency 48038.955 ms 2024-04-23T18:27:27.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.33 MB/sec execute 212 sec latency 49039.049 ms 2024-04-23T18:27:28.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:28 smithi172 ceph-mon[32456]: pgmap v214: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T18:27:28.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:28 smithi173 ceph-mon[32906]: pgmap v214: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T18:27:28.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:28 smithi037 ceph-mon[29404]: pgmap v214: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T18:27:28.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.31 MB/sec execute 213 sec latency 50039.188 ms 2024-04-23T18:27:29.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.30 MB/sec execute 214 sec latency 51039.313 ms 2024-04-23T18:27:30.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:30 smithi172 ceph-mon[32456]: pgmap v215: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:30.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:30 smithi173 ceph-mon[32906]: pgmap v215: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:30.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:30 smithi037 ceph-mon[29404]: pgmap v215: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:30.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.28 MB/sec execute 215 sec latency 52039.436 ms 2024-04-23T18:27:31.830 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.27 MB/sec execute 216 sec latency 53039.576 ms 2024-04-23T18:27:32.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:32 smithi172 ceph-mon[32456]: pgmap v216: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:32.607 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:32 smithi173 ceph-mon[32906]: pgmap v216: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:32.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:32 smithi037 ceph-mon[29404]: pgmap v216: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:32.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.25 MB/sec execute 217 sec latency 54039.698 ms 2024-04-23T18:27:33.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.24 MB/sec execute 218 sec latency 55039.792 ms 2024-04-23T18:27:34.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:34 smithi037 ceph-mon[29404]: pgmap v217: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:27:34.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:34 smithi172 ceph-mon[32456]: pgmap v217: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:27:34.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:34 smithi173 ceph-mon[32906]: pgmap v217: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-23T18:27:34.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.22 MB/sec execute 219 sec latency 56039.933 ms 2024-04-23T18:27:35.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.21 MB/sec execute 220 sec latency 57040.043 ms 2024-04-23T18:27:36.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:36 smithi172 ceph-mon[32456]: pgmap v218: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:36.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:36 smithi173 ceph-mon[32906]: pgmap v218: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:36.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:36 smithi037 ceph-mon[29404]: pgmap v218: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:36.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.19 MB/sec execute 221 sec latency 58040.145 ms 2024-04-23T18:27:37.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.18 MB/sec execute 222 sec latency 59040.267 ms 2024-04-23T18:27:38.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:38 smithi172 ceph-mon[32456]: pgmap v219: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:27:38.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:38 smithi173 ceph-mon[32906]: pgmap v219: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:27:38.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:38 smithi037 ceph-mon[29404]: pgmap v219: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:27:38.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.16 MB/sec execute 223 sec latency 60040.337 ms 2024-04-23T18:27:39.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.15 MB/sec execute 224 sec latency 61040.451 ms 2024-04-23T18:27:40.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:40 smithi172 ceph-mon[32456]: pgmap v220: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:40.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:40 smithi173 ceph-mon[32906]: pgmap v220: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:40.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:40 smithi037 ceph-mon[29404]: pgmap v220: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:40.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.14 MB/sec execute 225 sec latency 62040.586 ms 2024-04-23T18:27:41.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.12 MB/sec execute 226 sec latency 63040.656 ms 2024-04-23T18:27:42.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:42 smithi172 ceph-mon[32456]: pgmap v221: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:42.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:42 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:42.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:42 smithi173 ceph-mon[32906]: pgmap v221: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:42.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:42 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:42.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:42 smithi037 ceph-mon[29404]: pgmap v221: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T18:27:42.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:42 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:42.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370575 3.11 MB/sec execute 227 sec latency 64040.777 ms 2024-04-23T18:27:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe949ffb640 1 -- 172.21.15.37:0/2144853791 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556edda65200 con 0x556eddb7ed60 2024-04-23T18:27:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe949ffb640 10 client.16347 handle_command_reply: tid=0 2024-04-23T18:27:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 client.16347 shutdown 2024-04-23T18:27:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 msgr2=0x556eddb7d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 0x556eddb7d1f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0057c0 tx=0x7fe95c009bc0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 shutdown_connections 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x556eddb7ed60 0x556eddb81180 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556eddb03800 0x556eddb7d1f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556eddaf8f40 0x556eddafd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 --2- 172.21.15.37:0/2144853791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556eddaf8620 0x556eddafd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe949ffb640 0 client.16347 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 >> 172.21.15.37:0/2144853791 conn(0x556eddaf4b30 msgr2=0x556eddaf3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 shutdown_connections 2024-04-23T18:27:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 1 -- 172.21.15.37:0/2144853791 wait complete. 2024-04-23T18:27:43.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fba5b7fe640 1 -- 172.21.15.37:0/2870815819 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcaa8f2d00 con 0x55bcaa99f4d0 2024-04-23T18:27:43.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fba5b7fe640 10 client.16401 handle_command_reply: tid=0 2024-04-23T18:27:43.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fba8561a740 1 client.16401 shutdown 2024-04-23T18:27:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.562+0000 7fe96bd95740 20 client.16347 trim_cache size 0 max 16384 2024-04-23T18:27:43.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 msgr2=0x55bcaa98c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa98c090 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c00b990 comp rx=0 tx=0).stop 2024-04-23T18:27:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:27:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:27:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:27:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 shutdown_connections 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaa99f4d0 0x55bcaa9a18c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa98cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa980540 0x55bcaa98c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba5b7fe640 0 client.16401 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 --2- 172.21.15.37:0/2870815819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa98c090 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 >> 172.21.15.37:0/2870815819 conn(0x55bcaa97c040 msgr2=0x55bcaa981e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 shutdown_connections 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 1 -- 172.21.15.37:0/2870815819 wait complete. 2024-04-23T18:27:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fba8561a740 20 client.16401 trim_cache size 0 max 16384 2024-04-23T18:27:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe94c0a3dc0 msgr2=0x7fe94c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fe968fa8640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe94c0a3dc0 0x7fe94c0a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe95c010860 tx=0x7fe95c01d000 comp rx=0 tx=0).stop 2024-04-23T18:27:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 msgr2=0x7fe964198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.563+0000 7fe968fa8640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 0x7fe964198cb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d770 tx=0x7fe95800dc30 comp rx=0 tx=0).stop 2024-04-23T18:27:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 shutdown_connections 2024-04-23T18:27:43.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe94c0a3dc0 0x7fe94c0a6280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe96410c560 0x7fe96419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe964103b40 0x7fe964198cb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 --2- 172.21.15.37:0/1148802493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe964103220 0x7fe964198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 >> 172.21.15.37:0/1148802493 conn(0x7fe964077fd0 msgr2=0x7fe9640fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 shutdown_connections 2024-04-23T18:27:43.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:43.564+0000 7fe968fa8640 1 -- 172.21.15.37:0/1148802493 wait complete. 2024-04-23T18:27:43.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.593+0000 7fba8561a740 1 Processor -- start 2024-04-23T18:27:43.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.593+0000 7fba8561a740 1 -- start start 2024-04-23T18:27:43.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:43 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:43.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:43 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:43.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:43 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.593+0000 7fba8561a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa99ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.593+0000 7fba8561a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa980540 0x55bcaa9a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.593+0000 7fba8561a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa9a7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba8561a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1ae380 con 0x55bcaa97fc20 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba8561a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1abbf0 con 0x55bcaa98f120 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba8561a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1a9a40 con 0x55bcaa980540 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa99ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa99ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52588/0 (socket says 172.21.15.37:52588) 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba80ea4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa9a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 -- 172.21.15.37:0/3889480730 learned_addr learned my addr 172.21.15.37:0/3889480730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa980540 0x55bcaa9a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 -- 172.21.15.37:0/3889480730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa980540 msgr2=0x55bcaa9a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa980540 0x55bcaa9a6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 -- 172.21.15.37:0/3889480730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 msgr2=0x55bcaa9a7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa9a7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7bfff640 1 -- 172.21.15.37:0/3889480730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcaa98a5e0 con 0x55bcaa97fc20 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.594+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa980540 0x55bcaa9a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.595+0000 7fba7bfff640 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa99ae80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fba6c003c20 tx=0x7fba6c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974d958deff74bca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.595+0000 7fba61ffb640 1 -- 172.21.15.37:0/3889480730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c02a040 con 0x55bcaa97fc20 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.595+0000 7fba61ffb640 1 -- 172.21.15.37:0/3889480730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c0053a0 con 0x55bcaa97fc20 2024-04-23T18:27:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.595+0000 7fba61ffb640 1 -- 172.21.15.37:0/3889480730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0117e0 con 0x55bcaa97fc20 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 -- 172.21.15.37:0/3889480730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 msgr2=0x55bcaa99ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa99ae80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fba6c003c20 tx=0x7fba6c02f000 comp rx=0 tx=0).stop 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 -- 172.21.15.37:0/3889480730 shutdown_connections 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa98f120 0x55bcaa9a7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa980540 0x55bcaa9a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 --2- 172.21.15.37:0/3889480730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa97fc20 0x55bcaa99ae80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 -- 172.21.15.37:0/3889480730 >> 172.21.15.37:0/3889480730 conn(0x55bcaa97c040 msgr2=0x55bcaa97b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 -- 172.21.15.37:0/3889480730 shutdown_connections 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.596+0000 7fba8561a740 1 -- 172.21.15.37:0/3889480730 wait complete. 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 -- start start 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 Processor -- start 2024-04-23T18:27:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 0x55bcaa997390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa9978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa994360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1aa0b0 con 0x55bcaa980540 2024-04-23T18:27:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1ab520 con 0x55bcaa97fc20 2024-04-23T18:27:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1aa780 con 0x55bcaa98f120 2024-04-23T18:27:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba8561a740 10 client.0 _ll_register_callbacks cb 0x7fba6c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.597+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 0x55bcaa997390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa9978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba80ea4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa994360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 0x55bcaa997390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:35686/0 (socket says 172.21.15.37:35686) 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 -- 172.21.15.37:0/157491919 learned_addr learned my addr 172.21.15.37:0/157491919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 -- 172.21.15.37:0/157491919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 msgr2=0x55bcaa994360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa994360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 -- 172.21.15.37:0/157491919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 msgr2=0x55bcaa9978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa9978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 -- 172.21.15.37:0/157491919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcaa9949a0 con 0x55bcaa97fc20 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa9978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba7bfff640 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 0x55bcaa997390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b5a0 tx=0x7fba6c00fdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1ae0f580e45e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:27:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.598+0000 7fba5b7fe640 1 -- 172.21.15.37:0/157491919 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004ca0 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6c0053a0 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba5b7fe640 1 -- 172.21.15.37:0/157491919 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fba6c03c040 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba5b7fe640 1 -- 172.21.15.37:0/157491919 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba6c003800 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 10 client.25880 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bcaaa06100 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba5b7fe640 1 -- 172.21.15.37:0/157491919 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c0053a0 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba5b7fe640 1 -- 172.21.15.37:0/157491919 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fba6c011c00 con 0x55bcaa97fc20 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 10 client.25880 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 20 client.25880 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaaa06710 0x55bcaaa08b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 4 client.25880 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.b-ops-9f848727-1"}] 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.599+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/tmp/mds.b-ops-9f848727-1"}) -- 0x55bcaa8ecb70 con 0x55bcaaa06710 2024-04-23T18:27:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.600+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaaa06710 0x55bcaaa08b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.600+0000 7fba7b7fe640 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaaa06710 0x55bcaaa08b00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e678ce40f07aa055 server_cookie=ced133fe5617a7b3 in_seq=0 out_seq=0 2024-04-23T18:27:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.600+0000 7fba5b7fe640 10 client.25880 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.601+0000 7fba5b7fe640 1 -- 172.21.15.37:0/157491919 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcaa8ecb70 con 0x55bcaaa06710 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.601+0000 7fba5b7fe640 10 client.25880 handle_command_reply: tid=0 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.601+0000 7fba8561a740 1 client.25880 shutdown 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 msgr2=0x55bcaa997390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 0x55bcaa997390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b5a0 tx=0x7fba6c00fdd0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 shutdown_connections 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bcaaa06710 0x55bcaaa08b00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bcaa98f120 0x55bcaa994360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcaa980540 0x55bcaa9978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 --2- 172.21.15.37:0/157491919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcaa97fc20 0x55bcaa997390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba5b7fe640 0 client.25880 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.602+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 >> 172.21.15.37:0/157491919 conn(0x55bcaa97c040 msgr2=0x55bcaa97a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.603+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 shutdown_connections 2024-04-23T18:27:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.603+0000 7fba8561a740 1 -- 172.21.15.37:0/157491919 wait complete. 2024-04-23T18:27:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.603+0000 7fba8561a740 20 client.25880 trim_cache size 0 max 16384 2024-04-23T18:27:43.600 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:27:43.600 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/tmp/mds.b-ops-9f848727-1" 2024-04-23T18:27:43.600 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fba640a3d20 msgr2=0x7fba640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fba640a3d20 0x7fba640a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba74005c40 tx=0x7fba74009200 comp rx=0 tx=0).stop 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 msgr2=0x7fba7c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 0x7fba7c19e920 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fba70032fd0 tx=0x7fba70004270 comp rx=0 tx=0).stop 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 shutdown_connections 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fba640a3d20 0x7fba640a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba7c198590 0x7fba7c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba7c1072b0 0x7fba7c19e920 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 --2- 172.21.15.37:0/3227231554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba7c100af0 0x7fba7c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.604+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 >> 172.21.15.37:0/3227231554 conn(0x7fba7c0760e0 msgr2=0x7fba7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.605+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 shutdown_connections 2024-04-23T18:27:43.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.605+0000 7fba82907640 1 -- 172.21.15.37:0/3227231554 wait complete. 2024-04-23T18:27:43.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:43 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:43.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:43 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:43.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:43 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:27:43.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:43 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:43.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:43 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:43.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:43 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:27:43.618 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-23T18:27:43.634 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.LoBFzFpLVZ 2024-04-23T18:27:43.635 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /tmp/mds.h-ops-9f848727-1 /tmp/tmp.LoBFzFpLVZ 2024-04-23T18:27:43.699 INFO:teuthology.orchestra.run.smithi172.stderr:cp: cannot stat '/tmp/mds.h-ops-9f848727-1': No such file or directory 2024-04-23T18:27:43.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:27:43.701 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/tmp/mds.h-ops-9f848727-1' on rank 2 (h), error: Command failed on smithi172 with status 1: 'sudo cp /tmp/mds.h-ops-9f848727-1 /tmp/tmp.LoBFzFpLVZ' 2024-04-23T18:27:43.701 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-04-23T18:27:43.716 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.aRRCBrPvGA 2024-04-23T18:27:43.716 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /tmp/mds.i-ops-9f848727-1 /tmp/tmp.aRRCBrPvGA 2024-04-23T18:27:43.781 INFO:teuthology.orchestra.run.smithi173.stderr:cp: cannot stat '/tmp/mds.i-ops-9f848727-1': No such file or directory 2024-04-23T18:27:43.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:27:43.782 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/tmp/mds.i-ops-9f848727-1' on rank 1 (i), error: Command failed on smithi173 with status 1: 'sudo cp /tmp/mds.i-ops-9f848727-1 /tmp/tmp.aRRCBrPvGA' 2024-04-23T18:27:43.782 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-23T18:27:43.798 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.2xTDaBNVmg 2024-04-23T18:27:43.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /tmp/mds.b-ops-9f848727-1 /tmp/tmp.2xTDaBNVmg 2024-04-23T18:27:43.831 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370745 3.09 MB/sec execute 228 sec latency 64757.685 ms 2024-04-23T18:27:43.861 INFO:teuthology.orchestra.run.smithi172.stderr:cp: cannot stat '/tmp/mds.b-ops-9f848727-1': No such file or directory 2024-04-23T18:27:43.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:27:43.862 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/tmp/mds.b-ops-9f848727-1' on rank 0 (b), error: Command failed on smithi172 with status 1: 'sudo cp /tmp/mds.b-ops-9f848727-1 /tmp/tmp.2xTDaBNVmg' 2024-04-23T18:27:43.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 -- 172.21.15.37:0/4045956789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103230 msgr2=0x7f1150103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 --2- 172.21.15.37:0/4045956789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103230 0x7f1150103610 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1144009900 tx=0x7f114402a480 comp rx=0 tx=0).stop 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 -- 172.21.15.37:0/4045956789 shutdown_connections 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 --2- 172.21.15.37:0/4045956789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115010c570 0x7f115010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 --2- 172.21.15.37:0/4045956789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103b50 0x7f115010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 --2- 172.21.15.37:0/4045956789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103230 0x7f1150103610 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 -- 172.21.15.37:0/4045956789 >> 172.21.15.37:0/4045956789 conn(0x7f1150077fb0 msgr2=0x7f11500783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:43.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 -- 172.21.15.37:0/4045956789 shutdown_connections 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.977+0000 7f11571ad640 1 -- 172.21.15.37:0/4045956789 wait complete. 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 Processor -- start 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 -- start start 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103230 0x7f1150100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 0x7f11501013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115010c570 0x7f1150101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11501121d0 con 0x7f1150103b50 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1150112050 con 0x7f115010c570 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f11571ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1150112350 con 0x7f1150103230 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f114ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 0x7f11501013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f114ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 0x7f11501013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52620/0 (socket says 172.21.15.37:52620) 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f114ffff640 1 -- 172.21.15.37:0/3973467129 learned_addr learned my addr 172.21.15.37:0/3973467129 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.978+0000 7f115574a640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115010c570 0x7f1150101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f1154f49640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103230 0x7f1150100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114ffff640 1 -- 172.21.15.37:0/3973467129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103230 msgr2=0x7f1150100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114ffff640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103230 0x7f1150100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114ffff640 1 -- 172.21.15.37:0/3973467129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115010c570 msgr2=0x7f1150101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114ffff640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115010c570 0x7f1150101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114ffff640 1 -- 172.21.15.37:0/3973467129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11501ab110 con 0x7f1150103b50 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f1154f49640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103230 0x7f1150100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114ffff640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 0x7f11501013c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e9f0 tx=0x7f113c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00cd80 con 0x7f1150103b50 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11501ab340 con 0x7f1150103b50 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f113c0054d0 con 0x7f1150103b50 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c010690 con 0x7f1150103b50 2024-04-23T18:27:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.979+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11501ab840 con 0x7f1150103b50 2024-04-23T18:27:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.980+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115006ad80 con 0x7f1150103b50 2024-04-23T18:27:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.981+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f113c004e90 con 0x7f1150103b50 2024-04-23T18:27:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.982+0000 7f114dffb640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11400a3d20 0x7f11400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:43.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.982+0000 7f1154f49640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11400a3d20 0x7f11400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.982+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f113c0d0cc0 con 0x7f1150103b50 2024-04-23T18:27:43.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.983+0000 7f1154f49640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11400a3d20 0x7f11400a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1144031440 tx=0x7f1144005a20 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:43.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:43.987+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f113c09d350 con 0x7f1150103b50 2024-04-23T18:27:44.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.216+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11501076f0 con 0x7f1150103b50 2024-04-23T18:27:44.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.217+0000 7f114dffb640 1 -- 172.21.15.37:0/3973467129 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11881 (secure 0 0 0) 0x7f113c09cc50 con 0x7f1150103b50 2024-04-23T18:27:44.211 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:27:44.211 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:26:42.565223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24479,24484,24509]},"id":1}]} 2024-04-23T18:27:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 34 2024-04-23T18:27:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.219+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11400a3d20 msgr2=0x7f11400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.219+0000 7f11571ad640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11400a3d20 0x7f11400a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1144031440 tx=0x7f1144005a20 comp rx=0 tx=0).stop 2024-04-23T18:27:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.219+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 msgr2=0x7f11501013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.219+0000 7f11571ad640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 0x7f11501013c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e9f0 tx=0x7f113c00edf0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.219+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 shutdown_connections 2024-04-23T18:27:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11400a3d20 0x7f11400a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115010c570 0x7f1150101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150103b50 0x7f11501013c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 --2- 172.21.15.37:0/3973467129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1150103230 0x7f1150100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 >> 172.21.15.37:0/3973467129 conn(0x7f1150077fb0 msgr2=0x7f11500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 shutdown_connections 2024-04-23T18:27:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.220+0000 7f11571ad640 1 -- 172.21.15.37:0/3973467129 wait complete. 2024-04-23T18:27:44.224 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 9f848727 --await-for 42.0' 2024-04-23T18:27:44.224 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id 9f848727 --await-for 42.0 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 -- 172.21.15.37:0/909783578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102b40 msgr2=0x7f3054106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 --2- 172.21.15.37:0/909783578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102b40 0x7f3054106b80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f303c009900 tx=0x7f303c02a410 comp rx=0 tx=0).stop 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 -- 172.21.15.37:0/909783578 shutdown_connections 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 --2- 172.21.15.37:0/909783578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3054107260 0x7f305410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 --2- 172.21.15.37:0/909783578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102b40 0x7f3054106b80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 --2- 172.21.15.37:0/909783578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3054102190 0x7f3054102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 -- 172.21.15.37:0/909783578 >> 172.21.15.37:0/909783578 conn(0x7f30540fd990 msgr2=0x7f30540ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 -- 172.21.15.37:0/909783578 shutdown_connections 2024-04-23T18:27:44.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.339+0000 7f3059669640 1 -- 172.21.15.37:0/909783578 wait complete. 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 Processor -- start 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 -- start start 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102190 0x7f305419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3054107260 0x7f305419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30541985b0 0x7f3054198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30541121a0 con 0x7f3054102190 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3054112020 con 0x7f30541985b0 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3059669640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3054112320 con 0x7f3054107260 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102190 0x7f305419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102190 0x7f305419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52630/0 (socket says 172.21.15.37:52630) 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f3052ffd640 1 -- 172.21.15.37:0/56665491 learned_addr learned my addr 172.21.15.37:0/56665491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.340+0000 7f30537fe640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30541985b0 0x7f3054198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f304bfff640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3054107260 0x7f305419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3052ffd640 1 -- 172.21.15.37:0/56665491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3054107260 msgr2=0x7f305419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3052ffd640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3054107260 0x7f305419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3052ffd640 1 -- 172.21.15.37:0/56665491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30541985b0 msgr2=0x7f3054198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3052ffd640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30541985b0 0x7f3054198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3052ffd640 1 -- 172.21.15.37:0/56665491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30541990d0 con 0x7f3054102190 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f30537fe640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30541985b0 0x7f3054198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3052ffd640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3054102190 0x7f305419e400 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f304400d770 tx=0x7f304400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3050ff9640 1 -- 172.21.15.37:0/56665491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044017070 con 0x7f3054102190 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3050ff9640 1 -- 172.21.15.37:0/56665491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30440054d0 con 0x7f3054102190 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3059669640 1 -- 172.21.15.37:0/56665491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30541993c0 con 0x7f3054102190 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3050ff9640 1 -- 172.21.15.37:0/56665491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044004c30 con 0x7f3054102190 2024-04-23T18:27:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.341+0000 7f3059669640 1 -- 172.21.15.37:0/56665491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30541a55c0 con 0x7f3054102190 2024-04-23T18:27:44.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.343+0000 7f3050ff9640 1 -- 172.21.15.37:0/56665491 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3044004e10 con 0x7f3054102190 2024-04-23T18:27:44.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.343+0000 7f305c377740 1 Processor -- start 2024-04-23T18:27:44.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.343+0000 7f305c377740 1 -- start start 2024-04-23T18:27:44.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f305c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d580de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f305c377740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f305c377740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f305c377740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5653d5773380 con 0x5653d580e360 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f305c377740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653d570fd90 con 0x5653d581cf60 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f305c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5653d576f340 con 0x5653d580da40 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52640/0 (socket says 172.21.15.37:52640) 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 -- 172.21.15.37:0/3772838727 learned_addr learned my addr 172.21.15.37:0/3772838727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3050ff9640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f30400a3d20 0x7f30400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f304bfff640 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f30537fe640 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d580de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f30537fe640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f30400a3d20 0x7f30400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3050ff9640 1 -- 172.21.15.37:0/56665491 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f30440d0390 con 0x7f3054102190 2024-04-23T18:27:44.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 -- 172.21.15.37:0/3772838727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 msgr2=0x5653d580de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d580de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 -- 172.21.15.37:0/3772838727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 msgr2=0x5653d581f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f3052ffd640 1 -- 172.21.15.37:0/3772838727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d573a4d0 con 0x5653d580e360 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.344+0000 7f304bfff640 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f3052ffd640 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581ca20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f304409c020 tx=0x7f3044009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7180ee464934b08d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f30537fe640 1 --2- 172.21.15.37:0/56665491 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f30400a3d20 0x7f30400a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f304c013b30 tx=0x7f304c005c60 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f30497fa640 1 -- 172.21.15.37:0/3772838727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044020050 con 0x5653d580e360 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f30497fa640 1 -- 172.21.15.37:0/3772838727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304409ca20 con 0x5653d580e360 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f30497fa640 1 -- 172.21.15.37:0/3772838727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440cdd50 con 0x5653d580e360 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f305c377740 1 -- 172.21.15.37:0/3772838727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 msgr2=0x5653d581ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.345+0000 7f305c377740 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581ca20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f304409c020 tx=0x7f3044009f90 comp rx=0 tx=0).stop 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- 172.21.15.37:0/3772838727 shutdown_connections 2024-04-23T18:27:44.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581ca20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 --2- 172.21.15.37:0/3772838727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d580de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- 172.21.15.37:0/3772838727 >> 172.21.15.37:0/3772838727 conn(0x5653d5809e80 msgr2=0x5653d580a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- 172.21.15.37:0/3772838727 shutdown_connections 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- 172.21.15.37:0/3772838727 wait complete. 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- start start 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 Processor -- start 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5819d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5653d581fd40 con 0x5653d580e360 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653d570d670 con 0x5653d581cf60 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5653d581fec0 con 0x5653d580da40 2024-04-23T18:27:44.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f305c377740 10 client.0 _ll_register_callbacks cb 0x5653d570d060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f304bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f304bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52650/0 (socket says 172.21.15.37:52650) 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f304bfff640 1 -- 172.21.15.37:0/845084991 learned_addr learned my addr 172.21.15.37:0/845084991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.346+0000 7f30537fe640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f3052ffd640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5819d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f304bfff640 1 -- 172.21.15.37:0/845084991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 msgr2=0x5653d5819d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f304bfff640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5819d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f304bfff640 1 -- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 msgr2=0x5653d581a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f304bfff640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f304bfff640 1 -- 172.21.15.37:0/845084991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d5551ae0 con 0x5653d580e360 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f3052ffd640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5819d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f30537fe640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f304bfff640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581a260 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f305406b5d0 tx=0x7f303c031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0ec082bcca2322 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f303a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f303a7fc640 1 -- 172.21.15.37:0/845084991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c035040 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f303a7fc640 1 -- 172.21.15.37:0/845084991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f303c004380 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f304409ca20 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f303a7fc640 1 -- 172.21.15.37:0/845084991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f303c005030 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f305c377740 10 client.16425 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5653d55b2270 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f303a7fc640 1 -- 172.21.15.37:0/845084991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304409ca20 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.347+0000 7f303a7fc640 1 -- 172.21.15.37:0/845084991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f303c0035f0 con 0x5653d580e360 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f305c377740 10 client.16425 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f305c377740 10 client.16425 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f305c377740 20 client.16425 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f305c377740 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d582d1e0 0x5653d582f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f305c377740 4 client.16425 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d5780da0 con 0x5653d582d1e0 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f3052ffd640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d582d1e0 0x5653d582f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f303a7fc640 10 client.16425 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.348+0000 7f3052ffd640 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d582d1e0 0x5653d582f5d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2559a2dbaa68d5b server_cookie=f90d9e53f5f891e2 in_seq=0 out_seq=0 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f303a7fc640 1 -- 172.21.15.37:0/845084991 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653d5780da0 con 0x5653d582d1e0 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f303a7fc640 10 client.16425 handle_command_reply: tid=0 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 client.16425 shutdown 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 msgr2=0x5653d581a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581a260 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f305406b5d0 tx=0x7f303c031950 comp rx=0 tx=0).stop 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 shutdown_connections 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d582d1e0 0x5653d582f5d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f303a7fc640 0 client.16425 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d581a260 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 --2- 172.21.15.37:0/845084991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5819d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 >> 172.21.15.37:0/845084991 conn(0x5653d5809e80 msgr2=0x5653d580fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 shutdown_connections 2024-04-23T18:27:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 1 -- 172.21.15.37:0/845084991 wait complete. 2024-04-23T18:27:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.349+0000 7f305c377740 20 client.16425 trim_cache size 0 max 16384 2024-04-23T18:27:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.366+0000 7f305c377740 1 Processor -- start 2024-04-23T18:27:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.366+0000 7f305c377740 1 -- start start 2024-04-23T18:27:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f305c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5828c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f305c377740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d5818c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f305c377740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d58097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f305c377740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ae3f0 con 0x5653d580e360 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f305c377740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30541abc60 con 0x5653d581cf60 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f305c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30541a9ad0 con 0x5653d580da40 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d5818c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d5818c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52656/0 (socket says 172.21.15.37:52656) 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 -- 172.21.15.37:0/1797346029 learned_addr learned my addr 172.21.15.37:0/1797346029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f30537fe640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d58097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 -- 172.21.15.37:0/1797346029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 msgr2=0x5653d5828c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f3052ffd640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5828c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5828c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 -- 172.21.15.37:0/1797346029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 msgr2=0x5653d58097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d58097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 -- 172.21.15.37:0/1797346029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d54453d0 con 0x5653d580e360 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f30537fe640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d58097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f3052ffd640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5828c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.367+0000 7f304bfff640 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d5818c60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f303c02a930 tx=0x7f303c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cf1005d70e0dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f3048ff9640 1 -- 172.21.15.37:0/1797346029 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c035070 con 0x5653d580e360 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f3048ff9640 1 -- 172.21.15.37:0/1797346029 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c003c00 con 0x5653d580e360 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f3048ff9640 1 -- 172.21.15.37:0/1797346029 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c051070 con 0x5653d580e360 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 -- 172.21.15.37:0/1797346029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 msgr2=0x5653d5818c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d5818c60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f303c02a930 tx=0x7f303c02a410 comp rx=0 tx=0).stop 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 -- 172.21.15.37:0/1797346029 shutdown_connections 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d58097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580e360 0x5653d5818c60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 --2- 172.21.15.37:0/1797346029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580da40 0x5653d5828c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 -- 172.21.15.37:0/1797346029 >> 172.21.15.37:0/1797346029 conn(0x5653d5809e80 msgr2=0x5653d581a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 -- 172.21.15.37:0/1797346029 shutdown_connections 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.368+0000 7f305c377740 1 -- 172.21.15.37:0/1797346029 wait complete. 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 -- start start 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 Processor -- start 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 0x5653d581dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580e360 0x5653d581e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30541aa120 con 0x5653d580da40 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ab590 con 0x5653d581cf60 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30541aa7f0 con 0x5653d580e360 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f305c377740 10 client.0 _ll_register_callbacks cb 0x7f303c0315e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 0x5653d581dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 0x5653d581dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52664/0 (socket says 172.21.15.37:52664) 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 -- 172.21.15.37:0/153958622 learned_addr learned my addr 172.21.15.37:0/153958622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f304bfff640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580e360 0x5653d581e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 -- 172.21.15.37:0/153958622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580e360 msgr2=0x5653d581e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580e360 0x5653d581e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 -- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 msgr2=0x5653d581e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 -- 172.21.15.37:0/153958622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d5701e50 con 0x5653d580da40 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f304bfff640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580e360 0x5653d581e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f3052ffd640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 0x5653d581dcc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30440d0360 tx=0x7f304400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e308dcc32c1982 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.369+0000 7f303a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f303a7fc640 1 -- 172.21.15.37:0/153958622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440b5060 con 0x5653d580da40 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f303a7fc640 1 -- 172.21.15.37:0/153958622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f30440cdd50 con 0x5653d580da40 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c003c00 con 0x5653d580da40 2024-04-23T18:27:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f303a7fc640 1 -- 172.21.15.37:0/153958622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3044020040 con 0x5653d580da40 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 10 client.16431 fetch_fsmap learned FSMap version 34 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5653d5893810 con 0x5653d580da40 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f303a7fc640 1 -- 172.21.15.37:0/153958622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c003c00 con 0x5653d580da40 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f303a7fc640 1 -- 172.21.15.37:0/153958622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f30440a4b00 con 0x5653d580da40 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 10 client.16431 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 10 client.16431 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 20 client.16431 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d5893b50 0x5653d5895f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 4 client.16431 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9f848727", "await_for": 42.0}] 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9f848727", "await_for": 42.0}) -- 0x5653d577ac10 con 0x5653d5893b50 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.370+0000 7f304bfff640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d5893b50 0x5653d5895f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f303a7fc640 10 client.16431 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f304bfff640 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d5893b50 0x5653d5895f80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ebe9b2daca37f5 server_cookie=81197466d7ee6918 in_seq=0 out_seq=0 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f303a7fc640 1 -- 172.21.15.37:0/153958622 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: -110 ) ==== 8+0+592 (crc 0 0 0) 0x5653d577ac10 con 0x5653d5893b50 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f303a7fc640 10 client.16431 handle_command_reply: tid=0 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 client.16431 shutdown 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 msgr2=0x5653d581dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 0x5653d581dcc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30440d0360 tx=0x7f304400b480 comp rx=0 tx=0).stop 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 shutdown_connections 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d5893b50 0x5653d5895f80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d581cf60 0x5653d581e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f303a7fc640 0 client.16431 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d580e360 0x5653d581e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 --2- 172.21.15.37:0/153958622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d580da40 0x5653d581dcc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 >> 172.21.15.37:0/153958622 conn(0x5653d5809e80 msgr2=0x5653d580f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 shutdown_connections 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 1 -- 172.21.15.37:0/153958622 wait complete. 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:27:44.371+0000 7f305c377740 20 client.16431 trim_cache size 0 max 16384 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 34, 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 26, 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "9f848727": { 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 26, 2024-04-23T18:27:44.372 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 5.2, 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "TIMEDOUT", 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.6, 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCING", 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 60.0 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:44.373 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:44.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:44.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:27:44.374 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:27:44.374 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-23T18:27:44.375 INFO:teuthology.orchestra.run.smithi037.stderr:Error ETIMEDOUT: 2024-04-23T18:27:44.375 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 110 (ETIMEDOUT), stdout: { "epoch": 34, "leader": 24509, "set_version": 26, "sets": { "9f848727": { "version": 26, "age_ref": 5.2, "state": { "name": "TIMEDOUT", "age": 0.0 }, "timeout": 60.0, "expiration": 84.6, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCING", "age": 60.0 } } } } } } 2024-04-23T18:27:44.375 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 337, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 292, in do_quiesce raise RuntimeError(f"Error quiescing root: {rcinfo}") RuntimeError: Error quiescing root: 110 (ETIMEDOUT) 2024-04-23T18:27:44.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:44 smithi172 ceph-mon[32456]: pgmap v222: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:27:44.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:44 smithi173 ceph-mon[32906]: pgmap v222: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:27:44.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:44 smithi037 ceph-mon[29404]: pgmap v222: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-23T18:27:44.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370745 3.08 MB/sec execute 229 sec latency 1181.685 ms 2024-04-23T18:27:45.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:45 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3973467129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:45.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:45 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3973467129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:45.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:45 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3973467129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:45.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370745 3.07 MB/sec execute 230 sec latency 2181.730 ms 2024-04-23T18:27:46.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:46 smithi172 ceph-mon[32456]: pgmap v223: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T18:27:46.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:46 smithi173 ceph-mon[32906]: pgmap v223: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T18:27:46.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:46 smithi037 ceph-mon[29404]: pgmap v223: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T18:27:46.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370745 3.05 MB/sec execute 231 sec latency 3181.829 ms 2024-04-23T18:27:47.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 371339 3.05 MB/sec execute 232 sec latency 3925.699 ms 2024-04-23T18:27:48.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:48 smithi172 ceph-mon[32456]: pgmap v224: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 40 KiB/s wr, 28 op/s 2024-04-23T18:27:48.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:48 smithi173 ceph-mon[32906]: pgmap v224: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 40 KiB/s wr, 28 op/s 2024-04-23T18:27:48.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:48 smithi037 ceph-mon[29404]: pgmap v224: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 40 KiB/s wr, 28 op/s 2024-04-23T18:27:48.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 372989 3.09 MB/sec execute 233 sec latency 12.980 ms 2024-04-23T18:27:49.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 374123 3.10 MB/sec execute 234 sec latency 233.718 ms 2024-04-23T18:27:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:50 smithi172 ceph-mon[32456]: pgmap v225: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-04-23T18:27:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:50 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:27:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:50 smithi172 ceph-mon[32456]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:27:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:50 smithi173 ceph-mon[32906]: pgmap v225: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-04-23T18:27:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:50 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:27:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:50 smithi173 ceph-mon[32906]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:27:50.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:50 smithi037 ceph-mon[29404]: pgmap v225: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-04-23T18:27:50.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:50 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:27:50.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:50 smithi037 ceph-mon[29404]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:27:50.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 374123 3.08 MB/sec execute 235 sec latency 1233.839 ms 2024-04-23T18:27:51.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 374123 3.07 MB/sec execute 236 sec latency 2233.932 ms 2024-04-23T18:27:52.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:52 smithi172 ceph-mon[32456]: pgmap v227: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T18:27:52.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:52 smithi173 ceph-mon[32906]: pgmap v227: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T18:27:52.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:52 smithi037 ceph-mon[29404]: pgmap v227: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T18:27:52.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 374849 3.07 MB/sec execute 237 sec latency 2971.277 ms 2024-04-23T18:27:53.832 INFO:tasks.workunit.client.0.smithi037.stdout: 1 376510 3.10 MB/sec execute 238 sec latency 11.663 ms 2024-04-23T18:27:54.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:54 smithi172 ceph-mon[32456]: pgmap v228: 97 pgs: 97 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 469 KiB/s wr, 79 op/s 2024-04-23T18:27:54.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:54 smithi173 ceph-mon[32906]: pgmap v228: 97 pgs: 97 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 469 KiB/s wr, 79 op/s 2024-04-23T18:27:54.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:54 smithi037 ceph-mon[29404]: pgmap v228: 97 pgs: 97 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 469 KiB/s wr, 79 op/s 2024-04-23T18:27:54.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 377666 3.11 MB/sec execute 239 sec latency 167.148 ms 2024-04-23T18:27:55.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 377666 3.10 MB/sec execute 240 sec latency 1167.288 ms 2024-04-23T18:27:56.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:56 smithi172 ceph-mon[32456]: pgmap v229: 97 pgs: 97 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 469 KiB/s wr, 79 op/s 2024-04-23T18:27:56.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:56 smithi172 ceph-mon[32456]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:27:56.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:56 smithi173 ceph-mon[32906]: pgmap v229: 97 pgs: 97 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 469 KiB/s wr, 79 op/s 2024-04-23T18:27:56.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:56 smithi173 ceph-mon[32906]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:27:56.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:56 smithi037 ceph-mon[29404]: pgmap v229: 97 pgs: 97 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 469 KiB/s wr, 79 op/s 2024-04-23T18:27:56.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:56 smithi037 ceph-mon[29404]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:27:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 -- 172.21.15.37:0/542111348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 msgr2=0x7f538810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 --2- 172.21.15.37:0/542111348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f538810ee60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5380009ca0 tx=0x7f538002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 -- 172.21.15.37:0/542111348 shutdown_connections 2024-04-23T18:27:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 --2- 172.21.15.37:0/542111348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f538810ee60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 --2- 172.21.15.37:0/542111348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5388102b70 0x7f5388106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 --2- 172.21.15.37:0/542111348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53881021c0 0x7f53881025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 -- 172.21.15.37:0/542111348 >> 172.21.15.37:0/542111348 conn(0x7f53880fda20 msgr2=0x7f53880ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 -- 172.21.15.37:0/542111348 shutdown_connections 2024-04-23T18:27:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 -- 172.21.15.37:0/542111348 wait complete. 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 Processor -- start 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.674+0000 7f538f09a640 1 -- start start 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538f09a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53881021c0 0x7f53881a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538f09a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f53881a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538f09a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f538819a690 0x7f538819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538f09a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5388112200 con 0x7f5388107290 2024-04-23T18:27:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538f09a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5388112080 con 0x7f538819a690 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538f09a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5388112380 con 0x7f53881021c0 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f53881a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f53881a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56550/0 (socket says 172.21.15.37:56550) 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 -- 172.21.15.37:0/33325643 learned_addr learned my addr 172.21.15.37:0/33325643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538d637640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f538819a690 0x7f538819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538ce36640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53881021c0 0x7f53881a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 -- 172.21.15.37:0/33325643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53881021c0 msgr2=0x7f53881a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53881021c0 0x7f53881a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 -- 172.21.15.37:0/33325643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f538819a690 msgr2=0x7f538819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f538819a690 0x7f538819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 -- 172.21.15.37:0/33325643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538819b290 con 0x7f5388107290 2024-04-23T18:27:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f538d637640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f538819a690 0x7f538819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5387fff640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f53881a0b10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f537c00d740 tx=0x7f537c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.675+0000 7f5385ffb640 1 -- 172.21.15.37:0/33325643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c017070 con 0x7f5388107290 2024-04-23T18:27:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.676+0000 7f5385ffb640 1 -- 172.21.15.37:0/33325643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537c00de50 con 0x7f5388107290 2024-04-23T18:27:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.676+0000 7f5385ffb640 1 -- 172.21.15.37:0/33325643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c004ce0 con 0x7f5388107290 2024-04-23T18:27:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.676+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53881a7370 con 0x7f5388107290 2024-04-23T18:27:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.676+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881a7720 con 0x7f5388107290 2024-04-23T18:27:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.678+0000 7f5385ffb640 1 -- 172.21.15.37:0/33325643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f537c004e80 con 0x7f5388107290 2024-04-23T18:27:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.678+0000 7f5391db1740 1 Processor -- start 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 -- start start 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037ab620 0x55fc037aba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037b6800 0x55fc037b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fc03710970 con 0x55fc037abf40 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fc036ad4a0 con 0x55fc037ab620 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5391db1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fc0370c930 con 0x55fc037b6800 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5385ffb640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f53700a3dc0 0x7f53700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56554/0 (socket says 172.21.15.37:56554) 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 -- 172.21.15.37:0/1232008163 learned_addr learned my addr 172.21.15.37:0/1232008163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538d637640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037ab620 0x55fc037aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5387fff640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037b6800 0x55fc037b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538d637640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f53700a3dc0 0x7f53700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 -- 172.21.15.37:0/1232008163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037b6800 msgr2=0x55fc037b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037b6800 0x55fc037b8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 -- 172.21.15.37:0/1232008163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037ab620 msgr2=0x55fc037aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037ab620 0x55fc037aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5385ffb640 1 -- 172.21.15.37:0/33325643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f537c0d0270 con 0x7f5388107290 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538ce36640 1 -- 172.21.15.37:0/1232008163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc03423210 con 0x55fc037abf40 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f538d637640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037ab620 0x55fc037aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.679+0000 7f5387fff640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037b6800 0x55fc037b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f538ce36640 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037b62c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f537800bcb0 tx=0x7f537800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b4f2c5fc8a3cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f538d637640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f53700a3dc0 0x7f53700a6280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5380009a60 tx=0x7f53800095c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f536effd640 1 -- 172.21.15.37:0/1232008163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378015070 con 0x55fc037abf40 2024-04-23T18:27:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f536effd640 1 -- 172.21.15.37:0/1232008163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53780118f0 con 0x55fc037abf40 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f536effd640 1 -- 172.21.15.37:0/1232008163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378010910 con 0x55fc037abf40 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f5391db1740 1 -- 172.21.15.37:0/1232008163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 msgr2=0x55fc037b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.680+0000 7f5391db1740 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037b62c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f537800bcb0 tx=0x7f537800f730 comp rx=0 tx=0).stop 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 -- 172.21.15.37:0/1232008163 shutdown_connections 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037b6800 0x55fc037b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037b62c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 --2- 172.21.15.37:0/1232008163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037ab620 0x55fc037aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 -- 172.21.15.37:0/1232008163 >> 172.21.15.37:0/1232008163 conn(0x55fc037a7b30 msgr2=0x55fc037a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 -- 172.21.15.37:0/1232008163 shutdown_connections 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 -- 172.21.15.37:0/1232008163 wait complete. 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 -- start start 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 Processor -- start 2024-04-23T18:27:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.681+0000 7f5391db1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5391db1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 0x55fc037b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5391db1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fc037b9430 con 0x55fc037b6800 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5391db1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fc036ab0b0 con 0x55fc037abf40 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5391db1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fc037b95b0 con 0x55fc037ab620 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5391db1740 10 client.0 _ll_register_callbacks cb 0x55fc03698d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 0x55fc037b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 0x55fc037b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56560/0 (socket says 172.21.15.37:56560) 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 -- 172.21.15.37:0/3042756878 learned_addr learned my addr 172.21.15.37:0/3042756878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5387fff640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538ce36640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 -- 172.21.15.37:0/3042756878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 msgr2=0x55fc037b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 -- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 msgr2=0x55fc037b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 -- 172.21.15.37:0/3042756878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc034ef620 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f538d637640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 0x55fc037b3950 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5380035000 tx=0x7f538002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a0877111b8bf1b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5367fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5367fff640 1 -- 172.21.15.37:0/3042756878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380003e00 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5367fff640 1 -- 172.21.15.37:0/3042756878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f5380004650 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.682+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53780118f0 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5367fff640 1 -- 172.21.15.37:0/3042756878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5380004850 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 10 client.16443 fetch_fsmap learned FSMap version 35 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fc037c53a0 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5367fff640 1 -- 172.21.15.37:0/3042756878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53780118f0 con 0x55fc037b6800 2024-04-23T18:27:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5367fff640 1 -- 172.21.15.37:0/3042756878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f53800305c0 con 0x55fc037b6800 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 10 client.16443 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 20 client.16443 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037c6860 0x55fc037c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 4 client.16443 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc0371e390 con 0x55fc037c6860 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.683+0000 7f538ce36640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037c6860 0x55fc037c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.684+0000 7f538ce36640 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037c6860 0x55fc037c8cb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652fbff8af0b575e server_cookie=621d3de0d31f7147 in_seq=0 out_seq=0 2024-04-23T18:27:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.684+0000 7f5367fff640 10 client.16443 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.684+0000 7f5367fff640 1 -- 172.21.15.37:0/3042756878 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc0371e390 con 0x55fc037c6860 2024-04-23T18:27:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5367fff640 10 client.16443 handle_command_reply: tid=0 2024-04-23T18:27:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 client.16443 shutdown 2024-04-23T18:27:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 msgr2=0x55fc037b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 0x55fc037b3950 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5380035000 tx=0x7f538002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 shutdown_connections 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037c6860 0x55fc037c8cb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037b6800 0x55fc037b3950 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5367fff640 0 client.16443 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 --2- 172.21.15.37:0/3042756878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 >> 172.21.15.37:0/3042756878 conn(0x55fc037a7b30 msgr2=0x55fc037ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 shutdown_connections 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 1 -- 172.21.15.37:0/3042756878 wait complete. 2024-04-23T18:27:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.685+0000 7f5391db1740 20 client.16443 trim_cache size 0 max 16384 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 Processor -- start 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 -- start start 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037a7b30 0x55fc037c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53881b26c0 con 0x55fc037abf40 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53881aff60 con 0x55fc037a7b30 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.700+0000 7f5391db1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53881addd0 con 0x55fc037ab620 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56568/0 (socket says 172.21.15.37:56568) 2024-04-23T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 -- 172.21.15.37:0/1463004264 learned_addr learned my addr 172.21.15.37:0/1463004264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538ce36640 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037a7b30 0x55fc037c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f5387fff640 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 -- 172.21.15.37:0/1463004264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 msgr2=0x55fc037a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 -- 172.21.15.37:0/1463004264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037a7b30 msgr2=0x55fc037c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037a7b30 0x55fc037c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 -- 172.21.15.37:0/1463004264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc037ac8e0 con 0x55fc037abf40 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538ce36640 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037a7b30 0x55fc037c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f538d637640 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037ce5c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5380005fa0 tx=0x7f53800047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d63192e94a4146 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f536e7fc640 1 -- 172.21.15.37:0/1463004264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53800305c0 con 0x55fc037abf40 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f536e7fc640 1 -- 172.21.15.37:0/1463004264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538004f070 con 0x55fc037abf40 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.701+0000 7f536e7fc640 1 -- 172.21.15.37:0/1463004264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538002f8c0 con 0x55fc037abf40 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 -- 172.21.15.37:0/1463004264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 msgr2=0x55fc037ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037ce5c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5380005fa0 tx=0x7f53800047e0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 -- 172.21.15.37:0/1463004264 shutdown_connections 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037abf40 0x55fc037ce5c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037ab620 0x55fc037a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 --2- 172.21.15.37:0/1463004264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037a7b30 0x55fc037c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 -- 172.21.15.37:0/1463004264 >> 172.21.15.37:0/1463004264 conn(0x55fc037b3ab0 msgr2=0x55fc037a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 -- 172.21.15.37:0/1463004264 shutdown_connections 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 -- 172.21.15.37:0/1463004264 wait complete. 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 -- start start 2024-04-23T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.702+0000 7f5391db1740 1 Processor -- start 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037a7b30 0x55fc037bc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 0x55fc037bc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53881ae420 con 0x55fc037ab620 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53881af890 con 0x55fc037abf40 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53881aeaf0 con 0x55fc037a7b30 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5391db1740 10 client.0 _ll_register_callbacks cb 0x7f5380031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 0x55fc037bc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 0x55fc037bc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56580/0 (socket says 172.21.15.37:56580) 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 -- 172.21.15.37:0/3637736254 learned_addr learned my addr 172.21.15.37:0/3637736254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f538ce36640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037a7b30 0x55fc037bc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f538d637640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 -- 172.21.15.37:0/3637736254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037a7b30 msgr2=0x55fc037bc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037a7b30 0x55fc037bc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 -- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 msgr2=0x55fc037bcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037bcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 -- 172.21.15.37:0/3637736254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0382be40 con 0x55fc037ab620 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f538d637640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5387fff640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 0x55fc037bc7d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f537c0985d0 tx=0x7f537c06ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6236449528c9bf0e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5367fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5367fff640 1 -- 172.21.15.37:0/3637736254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c020040 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.703+0000 7f5367fff640 1 -- 172.21.15.37:0/3637736254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f537c0a0120 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538004f070 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5367fff640 1 -- 172.21.15.37:0/3637736254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f537c0a0320 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 10 client.16449 fetch_fsmap learned FSMap version 35 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fc0382d130 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5367fff640 1 -- 172.21.15.37:0/3637736254 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538004f070 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5367fff640 1 -- 172.21.15.37:0/3637736254 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f537c096ce0 con 0x55fc037ab620 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 10 client.16449 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 20 client.16449 populate_metadata read hostname 'smithi037' 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037b6800 0x55fc0382f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 4 client.16449 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc03718200 con 0x55fc037b6800 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.704+0000 7f538ce36640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037b6800 0x55fc0382f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.705+0000 7f5367fff640 10 client.16449 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.705+0000 7f538ce36640 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037b6800 0x55fc0382f920 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d397430393520e server_cookie=52cf56ff89753fe5 in_seq=0 out_seq=0 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5367fff640 1 -- 172.21.15.37:0/3637736254 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc03718200 con 0x55fc037b6800 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5367fff640 10 client.16449 handle_command_reply: tid=0 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 client.16449 shutdown 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 msgr2=0x55fc037bc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 0x55fc037bc7d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f537c0985d0 tx=0x7f537c06ba50 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 shutdown_connections 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55fc037b6800 0x55fc0382f920 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc037abf40 0x55fc037bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc037ab620 0x55fc037bc7d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 --2- 172.21.15.37:0/3637736254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc037a7b30 0x55fc037bc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5367fff640 0 client.16449 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 >> 172.21.15.37:0/3637736254 conn(0x55fc037b3ab0 msgr2=0x55fc037a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 shutdown_connections 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 1 -- 172.21.15.37:0/3637736254 wait complete. 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.738+0000 7f5391db1740 20 client.16449 trim_cache size 0 max 16384 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:27:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:27:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f53700a3dc0 msgr2=0x7f53700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f53700a3dc0 0x7f53700a6280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5380009a60 tx=0x7f53800095c0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 msgr2=0x7f53881a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f53881a0b10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f537c00d740 tx=0x7f537c00dc00 comp rx=0 tx=0).stop 2024-04-23T18:27:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 shutdown_connections 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f53700a3dc0 0x7f53700a6280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f538819a690 0x7f538819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5388107290 0x7f53881a0b10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 --2- 172.21.15.37:0/33325643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53881021c0 0x7f53881a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 >> 172.21.15.37:0/33325643 conn(0x7f53880fda20 msgr2=0x7f53880fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 shutdown_connections 2024-04-23T18:27:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:27:56.739+0000 7f538f09a640 1 -- 172.21.15.37:0/33325643 wait complete. 2024-04-23T18:27:56.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 377981 3.09 MB/sec execute 241 sec latency 2061.528 ms 2024-04-23T18:27:57.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 379954 3.13 MB/sec execute 242 sec latency 12.995 ms 2024-04-23T18:27:58.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:27:58 smithi172 ceph-mon[32456]: pgmap v231: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1010 KiB/s wr, 128 op/s 2024-04-23T18:27:58.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:27:58 smithi173 ceph-mon[32906]: pgmap v231: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1010 KiB/s wr, 128 op/s 2024-04-23T18:27:58.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:27:58 smithi037 ceph-mon[29404]: pgmap v231: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1010 KiB/s wr, 128 op/s 2024-04-23T18:27:58.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 381255 3.14 MB/sec execute 243 sec latency 18.704 ms 2024-04-23T18:27:59.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 381255 3.13 MB/sec execute 244 sec latency 1011.767 ms 2024-04-23T18:28:00.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:00 smithi172 ceph-mon[32456]: pgmap v232: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 853 KiB/s wr, 108 op/s 2024-04-23T18:28:00.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:00 smithi173 ceph-mon[32906]: pgmap v232: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 853 KiB/s wr, 108 op/s 2024-04-23T18:28:00.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:00 smithi037 ceph-mon[29404]: pgmap v232: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 853 KiB/s wr, 108 op/s 2024-04-23T18:28:00.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 381255 3.11 MB/sec execute 245 sec latency 2011.873 ms 2024-04-23T18:28:01.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 381255 3.10 MB/sec execute 246 sec latency 3011.950 ms 2024-04-23T18:28:02.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:02 smithi172 ceph-mon[32456]: pgmap v233: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 808 KiB/s wr, 102 op/s 2024-04-23T18:28:02.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:02 smithi173 ceph-mon[32906]: pgmap v233: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 808 KiB/s wr, 102 op/s 2024-04-23T18:28:02.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:02 smithi037 ceph-mon[29404]: pgmap v233: 97 pgs: 97 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 808 KiB/s wr, 102 op/s 2024-04-23T18:28:02.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 381941 3.10 MB/sec execute 247 sec latency 3759.451 ms 2024-04-23T18:28:03.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 383582 3.13 MB/sec execute 248 sec latency 33.840 ms 2024-04-23T18:28:04.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:04 smithi172 ceph-mon[32456]: pgmap v234: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 798 KiB/s wr, 113 op/s 2024-04-23T18:28:04.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:04 smithi173 ceph-mon[32906]: pgmap v234: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 798 KiB/s wr, 113 op/s 2024-04-23T18:28:04.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:04 smithi037 ceph-mon[29404]: pgmap v234: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 798 KiB/s wr, 113 op/s 2024-04-23T18:28:04.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 384763 3.14 MB/sec execute 249 sec latency 19.969 ms 2024-04-23T18:28:05.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:05 smithi172 ceph-mon[32456]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:28:05.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:05 smithi173 ceph-mon[32906]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:28:05.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:05 smithi037 ceph-mon[29404]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:28:05.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 384796 3.13 MB/sec execute 250 sec latency 984.556 ms 2024-04-23T18:28:06.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:06 smithi172 ceph-mon[32456]: pgmap v236: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 914 KiB/s wr, 129 op/s 2024-04-23T18:28:06.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:06 smithi173 ceph-mon[32906]: pgmap v236: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 914 KiB/s wr, 129 op/s 2024-04-23T18:28:06.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:06 smithi037 ceph-mon[29404]: pgmap v236: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 914 KiB/s wr, 129 op/s 2024-04-23T18:28:06.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 384796 3.12 MB/sec execute 251 sec latency 1984.650 ms 2024-04-23T18:28:07.833 INFO:tasks.workunit.client.0.smithi037.stdout: 1 384796 3.10 MB/sec execute 252 sec latency 2984.768 ms 2024-04-23T18:28:08.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:08 smithi172 ceph-mon[32456]: pgmap v237: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:08.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:08 smithi173 ceph-mon[32906]: pgmap v237: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:08.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:08 smithi037 ceph-mon[29404]: pgmap v237: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:08.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 385475 3.10 MB/sec execute 253 sec latency 3702.557 ms 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 -- 172.21.15.37:0/3323722327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11981011a0 msgr2=0x7f1198101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 --2- 172.21.15.37:0/3323722327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11981011a0 0x7f1198101620 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002a400 comp rx=0 tx=0).stop 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 -- 172.21.15.37:0/3323722327 shutdown_connections 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 --2- 172.21.15.37:0/3323722327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1198101b60 0x7f119810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 --2- 172.21.15.37:0/3323722327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11981011a0 0x7f1198101620 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 --2- 172.21.15.37:0/3323722327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1198106fb0 0x7f1198107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 -- 172.21.15.37:0/3323722327 >> 172.21.15.37:0/3323722327 conn(0x7f1198076c50 msgr2=0x7f1198077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 -- 172.21.15.37:0/3323722327 shutdown_connections 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.832+0000 7f119d35d640 1 -- 172.21.15.37:0/3323722327 wait complete. 2024-04-23T18:28:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 Processor -- start 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 -- start start 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11981011a0 0x7f1198197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 0x7f1198197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1198106fb0 0x7f1198197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1198111090 con 0x7f1198101b60 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1198110f10 con 0x7f1198106fb0 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f119d35d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1198111210 con 0x7f11981011a0 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f118ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 0x7f1198197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f118ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 0x7f1198197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54710/0 (socket says 172.21.15.37:54710) 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f118ffff640 1 -- 172.21.15.37:0/1529860319 learned_addr learned my addr 172.21.15.37:0/1529860319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11981011a0 0x7f1198197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.833+0000 7f11977fe640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1198106fb0 0x7f1198197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f118ffff640 1 -- 172.21.15.37:0/1529860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11981011a0 msgr2=0x7f1198197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f118ffff640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11981011a0 0x7f1198197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f118ffff640 1 -- 172.21.15.37:0/1529860319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1198106fb0 msgr2=0x7f1198197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f118ffff640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1198106fb0 0x7f1198197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f118ffff640 1 -- 172.21.15.37:0/1529860319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11981a6ab0 con 0x7f1198101b60 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f118ffff640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 0x7f1198197780 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1180005940 tx=0x7f1180004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f1194ff9640 1 -- 172.21.15.37:0/1529860319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118003f030 con 0x7f1198101b60 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11981a6d40 con 0x7f1198101b60 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f1194ff9640 1 -- 172.21.15.37:0/1529860319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1180003aa0 con 0x7f1198101b60 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f1194ff9640 1 -- 172.21.15.37:0/1529860319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180035070 con 0x7f1198101b60 2024-04-23T18:28:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.834+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981a7220 con 0x7f1198101b60 2024-04-23T18:28:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.836+0000 7f1194ff9640 1 -- 172.21.15.37:0/1529860319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1180042030 con 0x7f1198101b60 2024-04-23T18:28:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.836+0000 7f11a0078740 1 Processor -- start 2024-04-23T18:28:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.836+0000 7f11a0078740 1 -- start start 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11a0078740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cf620 0x5582921cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11a0078740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11a0078740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921dcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11a0078740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558292134970 con 0x5582921cff40 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11a0078740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582920d14a0 con 0x5582921cf620 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11a0078740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558292130930 con 0x5582921da800 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f118ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921dcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cf620 0x5582921cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f118ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921dcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:45290/0 (socket says 172.21.15.37:45290) 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f118ffff640 1 -- 172.21.15.37:0/3417007045 learned_addr learned my addr 172.21.15.37:0/3417007045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1194ff9640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11840a3dc0 0x7f11840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1196ffd640 1 -- 172.21.15.37:0/3417007045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 msgr2=0x5582921dcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1196ffd640 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921dcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1196ffd640 1 -- 172.21.15.37:0/3417007045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cf620 msgr2=0x5582921cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1196ffd640 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cf620 0x5582921cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f1196ffd640 1 -- 172.21.15.37:0/3417007045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558291e47210 con 0x5582921cff40 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f118ffff640 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921dcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.837+0000 7f11977fe640 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cf620 0x5582921cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.838+0000 7f1194ff9640 1 -- 172.21.15.37:0/1529860319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7f11800f0ea0 con 0x7f1198101b60 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.838+0000 7f11977fe640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11840a3dc0 0x7f11840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.838+0000 7f1196ffd640 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921da2c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f11880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864f9d8a10a37d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.838+0000 7f118d7fa640 1 -- 172.21.15.37:0/3417007045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118801c480 con 0x5582921cff40 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.838+0000 7f118d7fa640 1 -- 172.21.15.37:0/3417007045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118801cad0 con 0x5582921cff40 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.838+0000 7f118d7fa640 1 -- 172.21.15.37:0/3417007045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188011af0 con 0x5582921cff40 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 -- 172.21.15.37:0/3417007045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 msgr2=0x5582921da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11977fe640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11840a3dc0 0x7f11840a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f119806b860 tx=0x7f1190004a40 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921da2c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f11880107a0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 -- 172.21.15.37:0/3417007045 shutdown_connections 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921dcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921da2c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 --2- 172.21.15.37:0/3417007045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cf620 0x5582921cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 -- 172.21.15.37:0/3417007045 >> 172.21.15.37:0/3417007045 conn(0x5582921cbb30 msgr2=0x5582921cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 -- 172.21.15.37:0/3417007045 shutdown_connections 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 -- 172.21.15.37:0/3417007045 wait complete. 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 -- start start 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.839+0000 7f11a0078740 1 Processor -- start 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5582921dd430 con 0x5582921cff40 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582920cf0b0 con 0x5582921da800 2024-04-23T18:28:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5582921dd5b0 con 0x5582921cf620 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 10 client.0 _ll_register_callbacks cb 0x5582920bcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921d73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54738/0 (socket says 172.21.15.37:54738) 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 -- 172.21.15.37:0/193785171 learned_addr learned my addr 172.21.15.37:0/193785171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11977fe640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f1196ffd640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 -- 172.21.15.37:0/193785171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 msgr2=0x5582921da0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921da0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 -- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 msgr2=0x5582921d7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921d7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 -- 172.21.15.37:0/193785171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558291f13620 con 0x5582921cff40 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f118ffff640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921d73c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f11800eeb90 tx=0x7f1180031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af24af14e6351bea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f117e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f117e7fc640 1 -- 172.21.15.37:0/193785171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180049070 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f117e7fc640 1 -- 172.21.15.37:0/193785171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7f11800b9500 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.840+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118801cad0 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f117e7fc640 1 -- 172.21.15.37:0/193785171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f11800bc9d0 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 10 client.16461 fetch_fsmap learned FSMap version 36 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5582921e92d0 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f117e7fc640 1 -- 172.21.15.37:0/193785171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118801cad0 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f117e7fc640 1 -- 172.21.15.37:0/193785171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f11800eece0 con 0x5582921cff40 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 10 client.16461 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 20 client.16461 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x5582921ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 4 client.16461 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558292142390 con 0x5582921ea730 2024-04-23T18:28:09.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 387207 3.13 MB/sec execute 254 sec latency 11.321 ms 2024-04-23T18:28:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.841+0000 7f1196ffd640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x5582921ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.842+0000 7f117e7fc640 10 client.16461 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.842+0000 7f1196ffd640 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x5582921ecbb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8317314bc2f03455 server_cookie=40da97c61adb7a5b in_seq=0 out_seq=0 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f117e7fc640 1 -- 172.21.15.37:0/193785171 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558292142390 con 0x5582921ea730 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f117e7fc640 10 client.16461 handle_command_reply: tid=0 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 client.16461 shutdown 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 msgr2=0x5582921d73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921d73c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f11800eeb90 tx=0x7f1180031a10 comp rx=0 tx=0).stop 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 shutdown_connections 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x5582921ecbb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921d7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921d73c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f117e7fc640 0 client.16461 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 --2- 172.21.15.37:0/193785171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921da0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 >> 172.21.15.37:0/193785171 conn(0x5582921cbb30 msgr2=0x5582921d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 shutdown_connections 2024-04-23T18:28:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 1 -- 172.21.15.37:0/193785171 wait complete. 2024-04-23T18:28:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.843+0000 7f11a0078740 20 client.16461 trim_cache size 0 max 16384 2024-04-23T18:28:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 Processor -- start 2024-04-23T18:28:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 -- start start 2024-04-23T18:28:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 0x5582921d9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cff40 0x5582921f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11981b2640 con 0x5582921cf620 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11981aff10 con 0x5582921cff40 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11a0078740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11981add80 con 0x5582921da800 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 0x5582921d9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f1196ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 0x5582921d9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54740/0 (socket says 172.21.15.37:54740) 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f1196ffd640 1 -- 172.21.15.37:0/1604611429 learned_addr learned my addr 172.21.15.37:0/1604611429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f118ffff640 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cff40 0x5582921f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f11977fe640 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.858+0000 7f1196ffd640 1 -- 172.21.15.37:0/1604611429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 msgr2=0x5582921f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f1196ffd640 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f1196ffd640 1 -- 172.21.15.37:0/1604611429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cff40 msgr2=0x5582921f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f1196ffd640 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cff40 0x5582921f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f1196ffd640 1 -- 172.21.15.37:0/1604611429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582921d5910 con 0x5582921cf620 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f11977fe640 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f1196ffd640 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 0x5582921d9c00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1188010ca0 tx=0x7f118800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a9e3a5549ad9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f118cff9640 1 -- 172.21.15.37:0/1604611429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118802e070 con 0x5582921cf620 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f118cff9640 1 -- 172.21.15.37:0/1604611429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188005040 con 0x5582921cf620 2024-04-23T18:28:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.859+0000 7f118cff9640 1 -- 172.21.15.37:0/1604611429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118801c800 con 0x5582921cf620 2024-04-23T18:28:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- 172.21.15.37:0/1604611429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 msgr2=0x5582921d9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 0x5582921d9c00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1188010ca0 tx=0x7f118800b4a0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- 172.21.15.37:0/1604611429 shutdown_connections 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921da800 0x5582921f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921cff40 0x5582921f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- 172.21.15.37:0/1604611429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cf620 0x5582921d9c00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- 172.21.15.37:0/1604611429 >> 172.21.15.37:0/1604611429 conn(0x5582921cbb30 msgr2=0x5582921d7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- 172.21.15.37:0/1604611429 shutdown_connections 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- 172.21.15.37:0/1604611429 wait complete. 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- start start 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 Processor -- start 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921e0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11981ae3d0 con 0x5582921cff40 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11981af840 con 0x5582921da800 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11981aeaa0 con 0x5582921cf620 2024-04-23T18:28:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f11a0078740 10 client.0 _ll_register_callbacks cb 0x7f11800f46e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.860+0000 7f118ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921dfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54752/0 (socket says 172.21.15.37:54752) 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 -- 172.21.15.37:0/3260181746 learned_addr learned my addr 172.21.15.37:0/3260181746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f1196ffd640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f11977fe640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 -- 172.21.15.37:0/3260181746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 msgr2=0x5582921e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921e28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 -- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 msgr2=0x5582921e0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921e0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 -- 172.21.15.37:0/3260181746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582921e06c0 con 0x5582921cff40 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f118ffff640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921dfc40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1180003ec0 tx=0x7f1180005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fe3eb0819525ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f117e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f117e7fc640 1 -- 172.21.15.37:0/3260181746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800da060 con 0x5582921cff40 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f117e7fc640 1 -- 172.21.15.37:0/3260181746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7f11800037f0 con 0x5582921cff40 2024-04-23T18:28:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1188005040 con 0x5582921cff40 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f117e7fc640 1 -- 172.21.15.37:0/3260181746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1180049040 con 0x5582921cff40 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f11a0078740 10 client.16467 fetch_fsmap learned FSMap version 36 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.861+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558292250990 con 0x5582921cff40 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f117e7fc640 1 -- 172.21.15.37:0/3260181746 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188005040 con 0x5582921cff40 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f117e7fc640 1 -- 172.21.15.37:0/3260181746 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f11800eece0 con 0x5582921cff40 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f11a0078740 10 client.16467 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f11a0078740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f11a0078740 20 client.16467 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f11a0078740 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x558292253010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f11a0078740 4 client.16467 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55829213c200 con 0x5582921ea730 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f1196ffd640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x558292253010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f1196ffd640 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x558292253010 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9d14e2531bc42d server_cookie=81565cd6947ae9b7 in_seq=0 out_seq=0 2024-04-23T18:28:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.862+0000 7f117e7fc640 10 client.16467 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f117e7fc640 1 -- 172.21.15.37:0/3260181746 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55829213c200 con 0x5582921ea730 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f117e7fc640 10 client.16467 handle_command_reply: tid=0 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 client.16467 shutdown 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 msgr2=0x5582921dfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921dfc40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1180003ec0 tx=0x7f1180005c00 comp rx=0 tx=0).stop 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 shutdown_connections 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5582921ea730 0x558292253010 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582921da800 0x5582921e0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582921cff40 0x5582921dfc40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 --2- 172.21.15.37:0/3260181746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582921cf620 0x5582921e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f117e7fc640 0 client.16467 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 >> 172.21.15.37:0/3260181746 conn(0x5582921cbb30 msgr2=0x5582921ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 shutdown_connections 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 1 -- 172.21.15.37:0/3260181746 wait complete. 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.895+0000 7f11a0078740 20 client.16467 trim_cache size 0 max 16384 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:28:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:28:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11840a3dc0 msgr2=0x7f11840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11840a3dc0 0x7f11840a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f119806b860 tx=0x7f1190004a40 comp rx=0 tx=0).stop 2024-04-23T18:28:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 msgr2=0x7f1198197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 0x7f1198197780 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1180005940 tx=0x7f1180004270 comp rx=0 tx=0).stop 2024-04-23T18:28:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 shutdown_connections 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f11840a3dc0 0x7f11840a6280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1198106fb0 0x7f1198197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1198101b60 0x7f1198197780 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 --2- 172.21.15.37:0/1529860319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11981011a0 0x7f1198197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 >> 172.21.15.37:0/1529860319 conn(0x7f1198076c50 msgr2=0x7f11980ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 shutdown_connections 2024-04-23T18:28:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:09.896+0000 7f119d35d640 1 -- 172.21.15.37:0/1529860319 wait complete. 2024-04-23T18:28:10.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:10 smithi172 ceph-mon[32456]: pgmap v238: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:10.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:10 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:10.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:10 smithi173 ceph-mon[32906]: pgmap v238: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:10.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:10 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:10.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:10 smithi037 ceph-mon[29404]: pgmap v238: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:10.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:10 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:10.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 388368 3.14 MB/sec execute 255 sec latency 193.668 ms 2024-04-23T18:28:11.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 388368 3.13 MB/sec execute 256 sec latency 1193.788 ms 2024-04-23T18:28:12.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:11 smithi172 ceph-mon[32456]: pgmap v239: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:11 smithi172 ceph-mon[32456]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:28:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:11 smithi173 ceph-mon[32906]: pgmap v239: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:11 smithi173 ceph-mon[32906]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:28:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:11 smithi037 ceph-mon[29404]: pgmap v239: 97 pgs: 97 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 998 KiB/s wr, 133 op/s 2024-04-23T18:28:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:11 smithi037 ceph-mon[29404]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:28:12.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 389084 3.12 MB/sec execute 257 sec latency 1930.840 ms 2024-04-23T18:28:13.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 390822 3.16 MB/sec execute 258 sec latency 26.258 ms 2024-04-23T18:28:14.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:13 smithi172 ceph-mon[32456]: pgmap v241: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-23T18:28:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:13 smithi173 ceph-mon[32906]: pgmap v241: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-23T18:28:14.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:13 smithi037 ceph-mon[29404]: pgmap v241: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-04-23T18:28:14.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 391911 3.17 MB/sec execute 259 sec latency 367.169 ms 2024-04-23T18:28:15.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 391911 3.16 MB/sec execute 260 sec latency 1367.212 ms 2024-04-23T18:28:16.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:15 smithi172 ceph-mon[32456]: pgmap v242: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-23T18:28:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:15 smithi173 ceph-mon[32906]: pgmap v242: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-23T18:28:16.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:15 smithi037 ceph-mon[29404]: pgmap v242: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1018 KiB/s wr, 133 op/s 2024-04-23T18:28:16.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 391911 3.14 MB/sec execute 261 sec latency 2367.334 ms 2024-04-23T18:28:17.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 391911 3.13 MB/sec execute 262 sec latency 3367.407 ms 2024-04-23T18:28:18.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:17 smithi172 ceph-mon[32456]: pgmap v243: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:17 smithi173 ceph-mon[32906]: pgmap v243: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:18.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:17 smithi037 ceph-mon[29404]: pgmap v243: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:18.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 392627 3.13 MB/sec execute 263 sec latency 4087.492 ms 2024-04-23T18:28:19.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 394318 3.16 MB/sec execute 264 sec latency 24.624 ms 2024-04-23T18:28:20.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:19 smithi172 ceph-mon[32456]: pgmap v244: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:19 smithi173 ceph-mon[32906]: pgmap v244: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:20.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:19 smithi037 ceph-mon[29404]: pgmap v244: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:20.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 395459 3.17 MB/sec execute 265 sec latency 366.618 ms 2024-04-23T18:28:21.834 INFO:tasks.workunit.client.0.smithi037.stdout: 1 395459 3.16 MB/sec execute 266 sec latency 1366.689 ms 2024-04-23T18:28:22.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:21 smithi172 ceph-mon[32456]: pgmap v245: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:21 smithi173 ceph-mon[32906]: pgmap v245: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:22.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:21 smithi037 ceph-mon[29404]: pgmap v245: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 KiB/s wr, 112 op/s 2024-04-23T18:28:22.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 396176 3.15 MB/sec execute 267 sec latency 2103.592 ms 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 -- 172.21.15.37:0/2681484322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f541010f0 msgr2=0x7f7f541014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 --2- 172.21.15.37:0/2681484322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f541010f0 0x7f7f541014d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009900 tx=0x7f7f4402a410 comp rx=0 tx=0).stop 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 -- 172.21.15.37:0/2681484322 shutdown_connections 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 --2- 172.21.15.37:0/2681484322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f54106170 0x7f7f5410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 --2- 172.21.15.37:0/2681484322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f54101aa0 0x7f7f54105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 --2- 172.21.15.37:0/2681484322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f541010f0 0x7f7f541014d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 -- 172.21.15.37:0/2681484322 >> 172.21.15.37:0/2681484322 conn(0x7f7f54077fd0 msgr2=0x7f7f540783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 -- 172.21.15.37:0/2681484322 shutdown_connections 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 -- 172.21.15.37:0/2681484322 wait complete. 2024-04-23T18:28:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 Processor -- start 2024-04-23T18:28:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.992+0000 7f7f5a079640 1 -- start start 2024-04-23T18:28:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f5a079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 0x7f7f541986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f5a079640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f54106170 0x7f7f54198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f5a079640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 0x7f7f5419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f5a079640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54111100 con 0x7f7f5419cf80 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f5a079640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54110f80 con 0x7f7f541010f0 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f5a079640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f54111280 con 0x7f7f54106170 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f53fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 0x7f7f5419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 0x7f7f541986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f54106170 0x7f7f54198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 0x7f7f541986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:47902/0 (socket says 172.21.15.37:47902) 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 -- 172.21.15.37:0/3236828484 learned_addr learned my addr 172.21.15.37:0/3236828484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f53fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 0x7f7f5419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39490/0 (socket says 172.21.15.37:39490) 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 -- 172.21.15.37:0/3236828484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f54106170 msgr2=0x7f7f54198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f54106170 0x7f7f54198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 -- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 msgr2=0x7f7f5419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 0x7f7f5419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f537fe640 1 -- 172.21.15.37:0/3236828484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5419da70 con 0x7f7f541010f0 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.993+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 0x7f7f5419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.994+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 0x7f7f541986c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005940 tx=0x7f7f440040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.994+0000 7f7f517fa640 1 -- 172.21.15.37:0/3236828484 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4403f030 con 0x7f7f541010f0 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.994+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5419dd00 con 0x7f7f541010f0 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.994+0000 7f7f517fa640 1 -- 172.21.15.37:0/3236828484 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f440044c0 con 0x7f7f541010f0 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.994+0000 7f7f517fa640 1 -- 172.21.15.37:0/3236828484 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44035070 con 0x7f7f541010f0 2024-04-23T18:28:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.994+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f541a5520 con 0x7f7f541010f0 2024-04-23T18:28:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.996+0000 7f7f517fa640 1 -- 172.21.15.37:0/3236828484 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7f44042030 con 0x7f7f541010f0 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 Processor -- start 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f517fa640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f400a3dc0 0x7f7f400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 -- start start 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f400a3dc0 0x7f7f400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66b620 0x55a4ca66ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f517fa640 1 -- 172.21.15.37:0/3236828484 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f7f440f0b90 con 0x7f7f541010f0 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca6762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca678bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a4ca5d0970 con 0x55a4ca66bf40 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a4ca56d4a0 con 0x55a4ca66b620 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a4ca5cc930 con 0x55a4ca676800 2024-04-23T18:28:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca6762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca6762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39492/0 (socket says 172.21.15.37:39492) 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66b620 0x55a4ca66ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 -- 172.21.15.37:0/2394887009 learned_addr learned my addr 172.21.15.37:0/2394887009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f53fff640 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca678bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 -- 172.21.15.37:0/2394887009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 msgr2=0x55a4ca678bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca678bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 -- 172.21.15.37:0/2394887009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66b620 msgr2=0x55a4ca66ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66b620 0x55a4ca66ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.997+0000 7f7f4affd640 1 -- 172.21.15.37:0/2394887009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ca2e3210 con 0x55a4ca66bf40 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.998+0000 7f7f537fe640 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66b620 0x55a4ca66ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.998+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f400a3dc0 0x7f7f400a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c011f60 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.998+0000 7f7f4affd640 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca6762c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009be0 tx=0x7f7f3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594f898f84639430 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.998+0000 7f7f49ffb640 1 -- 172.21.15.37:0/2394887009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c004230 con 0x55a4ca66bf40 2024-04-23T18:28:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.998+0000 7f7f49ffb640 1 -- 172.21.15.37:0/2394887009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c009e50 con 0x55a4ca66bf40 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.998+0000 7f7f49ffb640 1 -- 172.21.15.37:0/2394887009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c019710 con 0x55a4ca66bf40 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/2394887009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 msgr2=0x55a4ca6762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca6762c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009be0 tx=0x7f7f3c011810 comp rx=0 tx=0).stop 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/2394887009 shutdown_connections 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca678bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca6762c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/2394887009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66b620 0x55a4ca66ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/2394887009 >> 172.21.15.37:0/2394887009 conn(0x55a4ca667b30 msgr2=0x55a4ca667f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/2394887009 shutdown_connections 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/2394887009 wait complete. 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 -- start start 2024-04-23T18:28:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:22.999+0000 7f7f5cd8f740 1 Processor -- start 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca676090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66bf40 0x55a4ca673380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca6738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a4ca679430 con 0x55a4ca66b620 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a4ca56b0b0 con 0x55a4ca66bf40 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a4ca6795b0 con 0x55a4ca676800 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 10 client.0 _ll_register_callbacks cb 0x55a4ca558d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca676090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca676090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39502/0 (socket says 172.21.15.37:39502) 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 -- 172.21.15.37:0/3317720066 learned_addr learned my addr 172.21.15.37:0/3317720066 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca6738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f4affd640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66bf40 0x55a4ca673380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 -- 172.21.15.37:0/3317720066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 msgr2=0x55a4ca6738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca6738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 -- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66bf40 msgr2=0x55a4ca673380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66bf40 0x55a4ca673380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 -- 172.21.15.37:0/3317720066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ca3af620 con 0x55a4ca66b620 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca6738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca676090 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7f440b9570 tx=0x7f7f440bcc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378952ac5300e7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f32ffd640 1 -- 172.21.15.37:0/3317720066 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44049050 con 0x55a4ca66b620 2024-04-23T18:28:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f32ffd640 1 -- 172.21.15.37:0/3317720066 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f7f440ee520 con 0x55a4ca66b620 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.000+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c009e50 con 0x55a4ca66b620 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f32ffd640 1 -- 172.21.15.37:0/3317720066 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7f440b9210 con 0x55a4ca66b620 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 10 client.16479 fetch_fsmap learned FSMap version 36 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a4ca6852e0 con 0x55a4ca66b620 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f32ffd640 1 -- 172.21.15.37:0/3317720066 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c009e50 con 0x55a4ca66b620 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f32ffd640 1 -- 172.21.15.37:0/3317720066 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f7f440c52d0 con 0x55a4ca66b620 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 10 client.16479 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 20 client.16479 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca686790 0x55a4ca688c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 4 client.16479 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4ca5de390 con 0x55a4ca686790 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.001+0000 7f7f4affd640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca686790 0x55a4ca688c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.002+0000 7f7f32ffd640 10 client.16479 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.002+0000 7f7f4affd640 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca686790 0x55a4ca688c00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55895b8d838f3dbd server_cookie=aa278a88eb54e254 in_seq=0 out_seq=0 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.002+0000 7f7f32ffd640 1 -- 172.21.15.37:0/3317720066 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4ca5de390 con 0x55a4ca686790 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.002+0000 7f7f32ffd640 10 client.16479 handle_command_reply: tid=0 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.002+0000 7f7f5cd8f740 1 client.16479 shutdown 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 msgr2=0x55a4ca676090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca676090 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7f440b9570 tx=0x7f7f440bcc20 comp rx=0 tx=0).stop 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 shutdown_connections 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca686790 0x55a4ca688c00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca676800 0x55a4ca6738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca66bf40 0x55a4ca673380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f32ffd640 0 client.16479 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3317720066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca676090 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 >> 172.21.15.37:0/3317720066 conn(0x55a4ca667b30 msgr2=0x55a4ca66d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 shutdown_connections 2024-04-23T18:28:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3317720066 wait complete. 2024-04-23T18:28:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.003+0000 7f7f5cd8f740 20 client.16479 trim_cache size 0 max 16384 2024-04-23T18:28:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 Processor -- start 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 -- start start 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca682320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66bf40 0x55a4ca66e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca675bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541b0420 con 0x55a4ca66b620 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541adcf0 con 0x55a4ca676800 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541abb60 con 0x55a4ca66bf40 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca682320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca682320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39512/0 (socket says 172.21.15.37:39512) 2024-04-23T18:28:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f537fe640 1 -- 172.21.15.37:0/3007412592 learned_addr learned my addr 172.21.15.37:0/3007412592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca675bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.018+0000 7f7f4affd640 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66bf40 0x55a4ca66e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f537fe640 1 -- 172.21.15.37:0/3007412592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66bf40 msgr2=0x55a4ca66e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66bf40 0x55a4ca66e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f537fe640 1 -- 172.21.15.37:0/3007412592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 msgr2=0x55a4ca675bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca675bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f537fe640 1 -- 172.21.15.37:0/3007412592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ca66bd70 con 0x55a4ca66b620 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f53fff640 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca675bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f537fe640 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca682320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7f440d3040 tx=0x7f7f440c58e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ea59fa49cb4946 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f497fa640 1 -- 172.21.15.37:0/3007412592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44049040 con 0x55a4ca66b620 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f497fa640 1 -- 172.21.15.37:0/3007412592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f440ee520 con 0x55a4ca66b620 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.019+0000 7f7f497fa640 1 -- 172.21.15.37:0/3007412592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440c5430 con 0x55a4ca66b620 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3007412592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 msgr2=0x55a4ca682320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca682320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7f440d3040 tx=0x7f7f440c58e0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3007412592 shutdown_connections 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca675bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66bf40 0x55a4ca66e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/3007412592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66b620 0x55a4ca682320 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3007412592 >> 172.21.15.37:0/3007412592 conn(0x55a4ca667b30 msgr2=0x55a4ca666cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3007412592 shutdown_connections 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/3007412592 wait complete. 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 -- start start 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 Processor -- start 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.020+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 0x55a4ca67ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca679f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca67a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541ac1b0 con 0x55a4ca66bf40 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541ad620 con 0x55a4ca676800 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f541ac880 con 0x55a4ca66b620 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 10 client.0 _ll_register_callbacks cb 0x7f7f440bc500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f4affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca679f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 0x55a4ca67ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 0x55a4ca67ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:60528/0 (socket says 172.21.15.37:60528) 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f537fe640 1 -- 172.21.15.37:0/1442598280 learned_addr learned my addr 172.21.15.37:0/1442598280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca67a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 -- 172.21.15.37:0/1442598280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 msgr2=0x55a4ca67ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 0x55a4ca67ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 -- 172.21.15.37:0/1442598280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 msgr2=0x55a4ca679f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca679f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 -- 172.21.15.37:0/1442598280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ca67ac90 con 0x55a4ca676800 2024-04-23T18:28:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f537fe640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 0x55a4ca67ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f4affd640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca679f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f53fff640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca67a4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55a4ca56b0f0 tx=0x7f7f4c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5a5f2a0ed234d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f32ffd640 1 -- 172.21.15.37:0/1442598280 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c01a040 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f440ee520 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.021+0000 7f7f32ffd640 1 -- 172.21.15.37:0/1442598280 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f7f4c020660 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f32ffd640 1 -- 172.21.15.37:0/1442598280 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c003e60 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 10 client.25946 fetch_fsmap learned FSMap version 36 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a4ca6ecd40 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f32ffd640 1 -- 172.21.15.37:0/1442598280 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f440ee520 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f32ffd640 1 -- 172.21.15.37:0/1442598280 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f7f4c030030 con 0x55a4ca676800 2024-04-23T18:28:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 10 client.25946 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 20 client.25946 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca6ed510 0x55a4ca6ef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 4 client.25946 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4ca5d8200 con 0x55a4ca6ed510 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.022+0000 7f7f537fe640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca6ed510 0x55a4ca6ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.023+0000 7f7f32ffd640 10 client.25946 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.023+0000 7f7f537fe640 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca6ed510 0x55a4ca6ef930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3dbd36c1abeba9c server_cookie=b7f7ffb8158ddd15 in_seq=0 out_seq=0 2024-04-23T18:28:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f32ffd640 1 -- 172.21.15.37:0/1442598280 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4ca5d8200 con 0x55a4ca6ed510 2024-04-23T18:28:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f32ffd640 10 client.25946 handle_command_reply: tid=0 2024-04-23T18:28:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 client.25946 shutdown 2024-04-23T18:28:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 msgr2=0x55a4ca67a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca67a4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55a4ca56b0f0 tx=0x7f7f4c004ab0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 shutdown_connections 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a4ca6ed510 0x55a4ca6ef930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4ca676800 0x55a4ca67a4b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a4ca66bf40 0x55a4ca679f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f32ffd640 0 client.25946 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 --2- 172.21.15.37:0/1442598280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4ca66b620 0x55a4ca67ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 >> 172.21.15.37:0/1442598280 conn(0x55a4ca667b30 msgr2=0x55a4ca666cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 shutdown_connections 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 1 -- 172.21.15.37:0/1442598280 wait complete. 2024-04-23T18:28:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.053+0000 7f7f5cd8f740 20 client.25946 trim_cache size 0 max 16384 2024-04-23T18:28:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:28:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:28:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:28:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f400a3dc0 msgr2=0x7f7f400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f400a3dc0 0x7f7f400a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c011f60 comp rx=0 tx=0).stop 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 msgr2=0x7f7f541986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 0x7f7f541986c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005940 tx=0x7f7f440040c0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 shutdown_connections 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f400a3dc0 0x7f7f400a6280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5419cf80 0x7f7f5419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f54106170 0x7f7f54198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 --2- 172.21.15.37:0/3236828484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f541010f0 0x7f7f541986c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 >> 172.21.15.37:0/3236828484 conn(0x7f7f54077fd0 msgr2=0x7f7f54069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 shutdown_connections 2024-04-23T18:28:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:23.054+0000 7f7f5a079640 1 -- 172.21.15.37:0/3236828484 wait complete. 2024-04-23T18:28:23.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 397879 3.18 MB/sec execute 268 sec latency 12.417 ms 2024-04-23T18:28:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:23 smithi172 ceph-mon[32456]: pgmap v246: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-04-23T18:28:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:23 smithi173 ceph-mon[32906]: pgmap v246: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-04-23T18:28:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:23 smithi037 ceph-mon[29404]: pgmap v246: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-04-23T18:28:24.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 399001 3.19 MB/sec execute 269 sec latency 385.624 ms 2024-04-23T18:28:25.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 399001 3.18 MB/sec execute 270 sec latency 1385.733 ms 2024-04-23T18:28:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:25 smithi172 ceph-mon[32456]: pgmap v247: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:28:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:25 smithi173 ceph-mon[32906]: pgmap v247: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:28:26.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:25 smithi037 ceph-mon[29404]: pgmap v247: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:28:26.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 399001 3.17 MB/sec execute 271 sec latency 2385.875 ms 2024-04-23T18:28:27.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 399001 3.16 MB/sec execute 272 sec latency 3385.979 ms 2024-04-23T18:28:28.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:27 smithi172 ceph-mon[32456]: pgmap v248: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-04-23T18:28:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:27 smithi173 ceph-mon[32906]: pgmap v248: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-04-23T18:28:28.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:27 smithi037 ceph-mon[29404]: pgmap v248: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 156 op/s 2024-04-23T18:28:28.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 399751 3.15 MB/sec execute 273 sec latency 4104.733 ms 2024-04-23T18:28:29.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 401452 3.18 MB/sec execute 274 sec latency 15.113 ms 2024-04-23T18:28:30.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:29 smithi172 ceph-mon[32456]: pgmap v249: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-04-23T18:28:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:29 smithi173 ceph-mon[32906]: pgmap v249: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-04-23T18:28:30.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:29 smithi037 ceph-mon[29404]: pgmap v249: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-04-23T18:28:30.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 402551 3.19 MB/sec execute 275 sec latency 363.285 ms 2024-04-23T18:28:31.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 402551 3.18 MB/sec execute 276 sec latency 1363.389 ms 2024-04-23T18:28:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:31 smithi172 ceph-mon[32456]: pgmap v250: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-04-23T18:28:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:31 smithi173 ceph-mon[32906]: pgmap v250: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-04-23T18:28:32.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:31 smithi037 ceph-mon[29404]: pgmap v250: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-04-23T18:28:32.835 INFO:tasks.workunit.client.0.smithi037.stdout: 1 403260 3.17 MB/sec execute 277 sec latency 2107.311 ms 2024-04-23T18:28:33.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 404932 3.21 MB/sec execute 278 sec latency 10.867 ms 2024-04-23T18:28:34.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:33 smithi172 ceph-mon[32456]: pgmap v251: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-04-23T18:28:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:33 smithi173 ceph-mon[32906]: pgmap v251: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-04-23T18:28:34.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:33 smithi037 ceph-mon[29404]: pgmap v251: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-04-23T18:28:34.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 406092 3.21 MB/sec execute 279 sec latency 325.839 ms 2024-04-23T18:28:35.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 406092 3.20 MB/sec execute 280 sec latency 1325.946 ms 2024-04-23T18:28:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 -- 172.21.15.37:0/2010396357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc102b70 msgr2=0x7f81dc106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 --2- 172.21.15.37:0/2010396357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc102b70 0x7f81dc106bb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009900 tx=0x7f81cc02a410 comp rx=0 tx=0).stop 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 -- 172.21.15.37:0/2010396357 shutdown_connections 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 --2- 172.21.15.37:0/2010396357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81dc107290 0x7f81dc10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 --2- 172.21.15.37:0/2010396357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc102b70 0x7f81dc106bb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 --2- 172.21.15.37:0/2010396357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc1021c0 0x7f81dc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 -- 172.21.15.37:0/2010396357 >> 172.21.15.37:0/2010396357 conn(0x7f81dc0fda20 msgr2=0x7f81dc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 -- 172.21.15.37:0/2010396357 shutdown_connections 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.153+0000 7f81e402a640 1 -- 172.21.15.37:0/2010396357 wait complete. 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 Processor -- start 2024-04-23T18:28:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 -- start start 2024-04-23T18:28:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc1021c0 0x7f81dc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81dc107290 0x7f81dc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 0x7f81dc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc112220 con 0x7f81dc19cf20 2024-04-23T18:28:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1120a0 con 0x7f81dc107290 2024-04-23T18:28:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e402a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1123a0 con 0x7f81dc1021c0 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.154+0000 7f81e25c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 0x7f81dc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 0x7f81dc19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49930/0 (socket says 172.21.15.37:49930) 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 -- 172.21.15.37:0/3825675106 learned_addr learned my addr 172.21.15.37:0/3825675106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc1021c0 0x7f81dc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e15c5640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81dc107290 0x7f81dc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 -- 172.21.15.37:0/3825675106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc1021c0 msgr2=0x7f81dc198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc1021c0 0x7f81dc198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 -- 172.21.15.37:0/3825675106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81dc107290 msgr2=0x7f81dc198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81dc107290 0x7f81dc198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 -- 172.21.15.37:0/3825675106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc19da10 con 0x7f81dc19cf20 2024-04-23T18:28:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e25c7640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 0x7f81dc19d3d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f81d400cce0 tx=0x7f81d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81d2ffd640 1 -- 172.21.15.37:0/3825675106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d400f450 con 0x7f81dc19cf20 2024-04-23T18:28:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc19dd00 con 0x7f81dc19cf20 2024-04-23T18:28:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81d2ffd640 1 -- 172.21.15.37:0/3825675106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d400faa0 con 0x7f81dc19cf20 2024-04-23T18:28:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81d2ffd640 1 -- 172.21.15.37:0/3825675106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4018b70 con 0x7f81dc19cf20 2024-04-23T18:28:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.155+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1a5470 con 0x7f81dc19cf20 2024-04-23T18:28:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81d2ffd640 1 -- 172.21.15.37:0/3825675106 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81d4018d10 con 0x7f81dc19cf20 2024-04-23T18:28:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 Processor -- start 2024-04-23T18:28:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 -- start start 2024-04-23T18:28:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024118620 0x564024118a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x5640241232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024123800 0x564024125bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81d2ffd640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f81c80a3dc0 0x7f81c80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56402407d970 con 0x564024118f40 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56402401a4a0 con 0x564024123800 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.158+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564024079930 con 0x564024118620 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x5640241232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e25c7640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f81c80a3dc0 0x7f81c80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x5640241232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49936/0 (socket says 172.21.15.37:49936) 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 -- 172.21.15.37:0/823390177 learned_addr learned my addr 172.21.15.37:0/823390177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e15c5640 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024118620 0x564024118a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e15c5640 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024123800 0x564024125bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81d2ffd640 1 -- 172.21.15.37:0/3825675106 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f81d40d0ba0 con 0x7f81dc19cf20 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 -- 172.21.15.37:0/823390177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024118620 msgr2=0x564024118a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024118620 0x564024118a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 -- 172.21.15.37:0/823390177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024123800 msgr2=0x564024125bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024123800 0x564024125bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 -- 172.21.15.37:0/823390177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564023d90210 con 0x564024118f40 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e25c7640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f81c80a3dc0 0x7f81c80a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81d40ce360 tx=0x7f81d401dfb0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x5640241232c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009d20 tx=0x7f81d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7735040e986ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81c7fff640 1 -- 172.21.15.37:0/823390177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d801e070 con 0x564024118f40 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81c7fff640 1 -- 172.21.15.37:0/823390177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d8009e10 con 0x564024118f40 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.159+0000 7f81c7fff640 1 -- 172.21.15.37:0/823390177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d80187a0 con 0x564024118f40 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 -- 172.21.15.37:0/823390177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 msgr2=0x5640241232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x5640241232c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009d20 tx=0x7f81d8011810 comp rx=0 tx=0).stop 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 -- 172.21.15.37:0/823390177 shutdown_connections 2024-04-23T18:28:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024123800 0x564024125bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x5640241232c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/823390177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024118620 0x564024118a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 -- 172.21.15.37:0/823390177 >> 172.21.15.37:0/823390177 conn(0x564024114b30 msgr2=0x564024114f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 -- 172.21.15.37:0/823390177 shutdown_connections 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.160+0000 7f81e6e1a740 1 -- 172.21.15.37:0/823390177 wait complete. 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 -- start start 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 Processor -- start 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x564024123180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024120470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x5640241209f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564024126430 con 0x564024118620 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5640240180b0 con 0x564024118f40 2024-04-23T18:28:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5640241265b0 con 0x564024123800 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e6e1a740 10 client.0 _ll_register_callbacks cb 0x564024005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e15c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x564024123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e15c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x564024123180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49940/0 (socket says 172.21.15.37:49940) 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e15c5640 1 -- 172.21.15.37:0/3543843244 learned_addr learned my addr 172.21.15.37:0/3543843244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.161+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024120470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x5640241209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 -- 172.21.15.37:0/3543843244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 msgr2=0x5640241209f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x5640241209f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 -- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 msgr2=0x564024120470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024120470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 -- 172.21.15.37:0/3543843244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564023e5c620 con 0x564024118620 2024-04-23T18:28:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e15c5640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x564024123180 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006010 tx=0x7f81cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e8a7fa2136cda6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81c57fa640 1 -- 172.21.15.37:0/3543843244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc035040 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81c57fa640 1 -- 172.21.15.37:0/3543843244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f81cc003d50 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d8009e10 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81c57fa640 1 -- 172.21.15.37:0/3543843244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f81cc003f50 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e6e1a740 10 client.16497 fetch_fsmap learned FSMap version 36 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.162+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564024132380 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81c57fa640 1 -- 172.21.15.37:0/3543843244 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d8009e10 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81c57fa640 1 -- 172.21.15.37:0/3543843244 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f81cc03d5e0 con 0x564024118620 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e6e1a740 10 client.16497 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e6e1a740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e6e1a740 20 client.16497 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5640241337f0 0x564024135c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e6e1a740 4 client.16497 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56402408b390 con 0x5640241337f0 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5640241337f0 0x564024135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81c57fa640 10 client.16497 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.163+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5640241337f0 0x564024135c00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a413eb4d9f41216 server_cookie=38be76025d7221ba in_seq=0 out_seq=0 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81c57fa640 1 -- 172.21.15.37:0/3543843244 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56402408b390 con 0x5640241337f0 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81c57fa640 10 client.16497 handle_command_reply: tid=0 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 client.16497 shutdown 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 msgr2=0x564024123180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x564024123180 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006010 tx=0x7f81cc009260 comp rx=0 tx=0).stop 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 shutdown_connections 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5640241337f0 0x564024135c00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x5640241209f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024120470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81c57fa640 0 client.16497 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/3543843244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x564024123180 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.164+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 >> 172.21.15.37:0/3543843244 conn(0x564024114b30 msgr2=0x564024113cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.165+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 shutdown_connections 2024-04-23T18:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.165+0000 7f81e6e1a740 1 -- 172.21.15.37:0/3543843244 wait complete. 2024-04-23T18:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.165+0000 7f81e6e1a740 20 client.16497 trim_cache size 0 max 16384 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 Processor -- start 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 -- start start 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x56402412f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b03f0 con 0x564024118620 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1adcc0 con 0x564024118f40 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.180+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1abb30 con 0x564024123800 2024-04-23T18:28:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x56402412f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x56402412f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49944/0 (socket says 172.21.15.37:49944) 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 -- 172.21.15.37:0/1937190237 learned_addr learned my addr 172.21.15.37:0/1937190237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 -- 172.21.15.37:0/1937190237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 msgr2=0x56402413b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402413b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 -- 172.21.15.37:0/1937190237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 msgr2=0x564024122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 -- 172.21.15.37:0/1937190237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564024118d70 con 0x564024118620 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x56402412f2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f81cc03da30 tx=0x7f81cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aea121f6f0a959 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81c77fe640 1 -- 172.21.15.37:0/1937190237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc058020 con 0x564024118620 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81c77fe640 1 -- 172.21.15.37:0/1937190237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81cc03dd20 con 0x564024118620 2024-04-23T18:28:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.181+0000 7f81c77fe640 1 -- 172.21.15.37:0/1937190237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc041070 con 0x564024118620 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1937190237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 msgr2=0x56402412f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x56402412f2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f81cc03da30 tx=0x7f81cc000bc0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1937190237 shutdown_connections 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118f40 0x564024122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1937190237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118620 0x56402412f2f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1937190237 >> 172.21.15.37:0/1937190237 conn(0x564024114b30 msgr2=0x564024113cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1937190237 shutdown_connections 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1937190237 wait complete. 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 -- start start 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.182+0000 7f81e6e1a740 1 Processor -- start 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118620 0x56402412bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x56402411cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402411d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1ac180 con 0x564024118f40 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1ad5f0 con 0x564024118620 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1ac850 con 0x564024123800 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e6e1a740 10 client.0 _ll_register_callbacks cb 0x7f81cc003750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x56402411cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x56402411cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49950/0 (socket says 172.21.15.37:49950) 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e15c5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118620 0x56402412bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 -- 172.21.15.37:0/1391195421 learned_addr learned my addr 172.21.15.37:0/1391195421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402411d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 -- 172.21.15.37:0/1391195421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 msgr2=0x56402411d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402411d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 -- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118620 msgr2=0x56402412bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118620 0x56402412bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 -- 172.21.15.37:0/1391195421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402411db60 con 0x564024118f40 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118620 0x56402412bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81e1dc6640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x56402411cdf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009c40 tx=0x7f81d8009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eed14a5421b16e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.183+0000 7f81c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81c57fa640 1 -- 172.21.15.37:0/1391195421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d801e040 con 0x564024118f40 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81c57fa640 1 -- 172.21.15.37:0/1391195421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f81d8009e10 con 0x564024118f40 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81cc03dd20 con 0x564024118f40 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81c57fa640 1 -- 172.21.15.37:0/1391195421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f81d8005330 con 0x564024118f40 2024-04-23T18:28:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 10 client.16503 fetch_fsmap learned FSMap version 36 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564024199dc0 con 0x564024118f40 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81c57fa640 1 -- 172.21.15.37:0/1391195421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81cc03dd20 con 0x564024118f40 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81c57fa640 1 -- 172.21.15.37:0/1391195421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f81d80035f0 con 0x564024118f40 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 10 client.16503 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 20 client.16503 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56402419a350 0x56402419c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 4 client.16503 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.184+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564024085200 con 0x56402419a350 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.185+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56402419a350 0x56402419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.185+0000 7f81c57fa640 10 client.16503 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.185+0000 7f81e15c5640 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56402419a350 0x56402419c740 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739cf9633effc019 server_cookie=59fb273987ad2604 in_seq=0 out_seq=0 2024-04-23T18:28:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81c57fa640 1 -- 172.21.15.37:0/1391195421 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564024085200 con 0x56402419a350 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81c57fa640 10 client.16503 handle_command_reply: tid=0 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 client.16503 shutdown 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 msgr2=0x56402411cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x56402411cdf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009c40 tx=0x7f81d8009a10 comp rx=0 tx=0).stop 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 shutdown_connections 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x56402419a350 0x56402419c740 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564024123800 0x56402411d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564024118f40 0x56402411cdf0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 --2- 172.21.15.37:0/1391195421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564024118620 0x56402412bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 >> 172.21.15.37:0/1391195421 conn(0x564024114b30 msgr2=0x56402411f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81c57fa640 0 client.16503 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 shutdown_connections 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 1 -- 172.21.15.37:0/1391195421 wait complete. 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.221+0000 7f81e6e1a740 20 client.16503 trim_cache size 0 max 16384 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:28:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f81c80a3dc0 msgr2=0x7f81c80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f81c80a3dc0 0x7f81c80a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81d40ce360 tx=0x7f81d401dfb0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 msgr2=0x7f81dc19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 0x7f81dc19d3d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f81d400cce0 tx=0x7f81d400a620 comp rx=0 tx=0).stop 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 shutdown_connections 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f81c80a3dc0 0x7f81c80a6280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81dc19cf20 0x7f81dc19d3d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81dc107290 0x7f81dc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 --2- 172.21.15.37:0/3825675106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc1021c0 0x7f81dc198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 >> 172.21.15.37:0/3825675106 conn(0x7f81dc0fda20 msgr2=0x7f81dc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 shutdown_connections 2024-04-23T18:28:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:36.222+0000 7f81e402a640 1 -- 172.21.15.37:0/3825675106 wait complete. 2024-04-23T18:28:36.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:35 smithi172 ceph-mon[32456]: pgmap v252: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-04-23T18:28:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:35 smithi173 ceph-mon[32906]: pgmap v252: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-04-23T18:28:36.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:35 smithi037 ceph-mon[29404]: pgmap v252: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-04-23T18:28:36.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 407943 3.23 MB/sec execute 281 sec latency 1688.486 ms 2024-04-23T18:28:37.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 409359 3.25 MB/sec execute 282 sec latency 14.111 ms 2024-04-23T18:28:38.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:37 smithi172 ceph-mon[32456]: pgmap v253: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-04-23T18:28:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:37 smithi173 ceph-mon[32906]: pgmap v253: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-04-23T18:28:38.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:37 smithi037 ceph-mon[29404]: pgmap v253: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-04-23T18:28:38.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 410403 3.24 MB/sec execute 283 sec latency 408.179 ms 2024-04-23T18:28:39.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 412094 3.27 MB/sec execute 284 sec latency 15.240 ms 2024-04-23T18:28:40.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:39 smithi172 ceph-mon[32456]: pgmap v254: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-04-23T18:28:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:39 smithi173 ceph-mon[32906]: pgmap v254: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-04-23T18:28:40.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:39 smithi037 ceph-mon[29404]: pgmap v254: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-04-23T18:28:40.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 413183 3.28 MB/sec execute 285 sec latency 401.115 ms 2024-04-23T18:28:41.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 413183 3.27 MB/sec execute 286 sec latency 1401.225 ms 2024-04-23T18:28:42.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:41 smithi172 ceph-mon[32456]: pgmap v255: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-04-23T18:28:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:41 smithi173 ceph-mon[32906]: pgmap v255: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-04-23T18:28:42.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:41 smithi037 ceph-mon[29404]: pgmap v255: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-04-23T18:28:42.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 413901 3.26 MB/sec execute 287 sec latency 2139.636 ms 2024-04-23T18:28:43.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:43 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:43.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:43 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:43.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:42 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:43.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415509 3.29 MB/sec execute 288 sec latency 11.074 ms 2024-04-23T18:28:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:44 smithi172 ceph-mon[32456]: pgmap v256: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-04-23T18:28:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:44 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:44 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:44 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:44.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:44 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:28:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:44 smithi173 ceph-mon[32906]: pgmap v256: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-04-23T18:28:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:44 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:44 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:44 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:44.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:44 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:28:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:44 smithi037 ceph-mon[29404]: pgmap v256: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-04-23T18:28:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:44 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:44 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:44 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:44.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:44 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:28:44.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 416732 3.30 MB/sec execute 289 sec latency 97.807 ms 2024-04-23T18:28:45.836 INFO:tasks.workunit.client.0.smithi037.stdout: 1 416732 3.29 MB/sec execute 290 sec latency 1097.859 ms 2024-04-23T18:28:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:46 smithi172 ceph-mon[32456]: pgmap v257: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-23T18:28:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:46 smithi173 ceph-mon[32906]: pgmap v257: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-23T18:28:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:46 smithi037 ceph-mon[29404]: pgmap v257: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 149 op/s 2024-04-23T18:28:46.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 416732 3.28 MB/sec execute 291 sec latency 2097.967 ms 2024-04-23T18:28:47.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 416732 3.27 MB/sec execute 292 sec latency 3098.038 ms 2024-04-23T18:28:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:48 smithi172 ceph-mon[32456]: pgmap v258: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-04-23T18:28:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:48 smithi173 ceph-mon[32906]: pgmap v258: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-04-23T18:28:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:48 smithi037 ceph-mon[29404]: pgmap v258: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2024-04-23T18:28:48.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 417470 3.26 MB/sec execute 293 sec latency 3817.494 ms 2024-04-23T18:28:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/3923701985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 msgr2=0x7f3ecc10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/3923701985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc10e370 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009900 tx=0x7f3ebc02a410 comp rx=0 tx=0).stop 2024-04-23T18:28:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/3923701985 shutdown_connections 2024-04-23T18:28:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/3923701985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc10e8b0 0x7f3ecc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/3923701985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc10e370 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/3923701985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ecc100d00 0x7f3ecc1010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/3923701985 >> 172.21.15.37:0/3923701985 conn(0x7f3ecc077fd0 msgr2=0x7f3ecc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/3923701985 shutdown_connections 2024-04-23T18:28:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.314+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/3923701985 wait complete. 2024-04-23T18:28:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 Processor -- start 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 -- start start 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc100d00 0x7f3ecc1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ecc10e8b0 0x7f3ecc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc114400 con 0x7f3ecc1016b0 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc114280 con 0x7f3ecc100d00 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.315+0000 7f3ed3b4f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc114580 con 0x7f3ecc10e8b0 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed18eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc100d00 0x7f3ecc1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56942/0 (socket says 172.21.15.37:56942) 2024-04-23T18:28:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed20ec640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ecc10e8b0 0x7f3ecc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 -- 172.21.15.37:0/1086524871 learned_addr learned my addr 172.21.15.37:0/1086524871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 -- 172.21.15.37:0/1086524871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ecc10e8b0 msgr2=0x7f3ecc19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ecc10e8b0 0x7f3ecc19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 -- 172.21.15.37:0/1086524871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc100d00 msgr2=0x7f3ecc1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc100d00 0x7f3ecc1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 -- 172.21.15.37:0/1086524871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc19afb0 con 0x7f3ecc1016b0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc1a0b60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005940 tx=0x7f3ebc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ec2ffd640 1 -- 172.21.15.37:0/1086524871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc03f030 con 0x7f3ecc1016b0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc19b240 con 0x7f3ecc1016b0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ec2ffd640 1 -- 172.21.15.37:0/1086524871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ebc0044c0 con 0x7f3ecc1016b0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ec2ffd640 1 -- 172.21.15.37:0/1086524871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035070 con 0x7f3ecc1016b0 2024-04-23T18:28:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.316+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc1a7470 con 0x7f3ecc1016b0 2024-04-23T18:28:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.319+0000 7f3ec2ffd640 1 -- 172.21.15.37:0/1086524871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ebc042030 con 0x7f3ecc1016b0 2024-04-23T18:28:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.319+0000 7f3ed686b740 1 Processor -- start 2024-04-23T18:28:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.319+0000 7f3ed686b740 1 -- start start 2024-04-23T18:28:49.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1837a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837f40 0x555ff18422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1842800 0x555ff1844bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed686b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555ff179c970 con 0x555ff1842800 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed686b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555ff17394a0 con 0x555ff1837f40 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed686b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ff1798930 con 0x555ff1837620 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed20ec640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1837a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed18eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837f40 0x555ff18422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1842800 0x555ff1844bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed20ec640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1837a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:60714/0 (socket says 172.21.15.37:60714) 2024-04-23T18:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed20ec640 1 -- 172.21.15.37:0/618374607 learned_addr learned my addr 172.21.15.37:0/618374607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ec2ffd640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f3eb80a3dc0 0x7f3eb80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed18eb640 1 -- 172.21.15.37:0/618374607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 msgr2=0x555ff1837a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed20ec640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f3eb80a3dc0 0x7f3eb80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1837a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed18eb640 1 -- 172.21.15.37:0/618374607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1842800 msgr2=0x555ff1844bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1842800 0x555ff1844bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ed18eb640 1 -- 172.21.15.37:0/618374607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff14af210 con 0x555ff1837f40 2024-04-23T18:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.320+0000 7f3ec2ffd640 1 -- 172.21.15.37:0/1086524871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f3ebc0f0b80 con 0x7f3ecc1016b0 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837f40 0x555ff18422c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009c00 tx=0x7f3ec8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3edc7c8cffab2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3ed20ec640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f3eb80a3dc0 0x7f3eb80a6280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005c40 tx=0x7f3ec4004ab0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3eb7fff640 1 -- 172.21.15.37:0/618374607 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8004230 con 0x555ff1837f40 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3eb7fff640 1 -- 172.21.15.37:0/618374607 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec80043d0 con 0x555ff1837f40 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3eb7fff640 1 -- 172.21.15.37:0/618374607 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8004c30 con 0x555ff1837f40 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3ed686b740 1 -- 172.21.15.37:0/618374607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837f40 msgr2=0x555ff18422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.321+0000 7f3ed686b740 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837f40 0x555ff18422c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009c00 tx=0x7f3ec8011810 comp rx=0 tx=0).stop 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 -- 172.21.15.37:0/618374607 shutdown_connections 2024-04-23T18:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1842800 0x555ff1844bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837f40 0x555ff18422c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 --2- 172.21.15.37:0/618374607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1837a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 -- 172.21.15.37:0/618374607 >> 172.21.15.37:0/618374607 conn(0x555ff1833b30 msgr2=0x555ff1833f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 -- 172.21.15.37:0/618374607 shutdown_connections 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 -- 172.21.15.37:0/618374607 wait complete. 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 -- start start 2024-04-23T18:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 Processor -- start 2024-04-23T18:28:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1842110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff183f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff183f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.322+0000 7f3ed686b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555ff1845430 con 0x555ff1837f40 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed686b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555ff17370b0 con 0x555ff1842800 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed686b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ff18455b0 con 0x555ff1837620 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed686b740 10 client.0 _ll_register_callbacks cb 0x555ff1724d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff183f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff183f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56946/0 (socket says 172.21.15.37:56946) 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2870311319 learned_addr learned my addr 172.21.15.37:0/2870311319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1842110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed20ec640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff183f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2870311319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 msgr2=0x555ff1842110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1842110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 msgr2=0x555ff183f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff183f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2870311319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff157b620 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff183f400 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0bc890 tx=0x7f3ebc0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6c9fdd19359fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2870311319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc049050 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2870311319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f3ebc003cf0 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec80043d0 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2870311319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc0a0450 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed686b740 10 client.16515 fetch_fsmap learned FSMap version 37 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.323+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555ff1851310 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2870311319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec80043d0 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2870311319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 37) ==== 8342+0+0 (secure 0 0 0) 0x7f3ebc0ee510 con 0x555ff1837f40 2024-04-23T18:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed686b740 10 client.16515 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed686b740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed686b740 20 client.16515 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff1852770 0x555ff1854bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed686b740 4 client.16515 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ff17aa390 con 0x555ff1852770 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.324+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff1852770 0x555ff1854bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff1852770 0x555ff1854bc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ec6dca1488ceb3 server_cookie=e3c63769c5aabcc5 in_seq=0 out_seq=0 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3eb57fa640 10 client.16515 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2870311319 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ff17aa390 con 0x555ff1852770 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3eb57fa640 10 client.16515 handle_command_reply: tid=0 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed686b740 1 client.16515 shutdown 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 msgr2=0x555ff183f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff183f400 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0bc890 tx=0x7f3ebc0092b0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 shutdown_connections 2024-04-23T18:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff1852770 0x555ff1854bc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.325+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff183f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3eb57fa640 0 client.16515 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff183f400 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2870311319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff1842110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 >> 172.21.15.37:0/2870311319 conn(0x555ff1833b30 msgr2=0x555ff1839860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 shutdown_connections 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3ed686b740 1 -- 172.21.15.37:0/2870311319 wait complete. 2024-04-23T18:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.326+0000 7f3ed686b740 20 client.16515 trim_cache size 0 max 16384 2024-04-23T18:28:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 Processor -- start 2024-04-23T18:28:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 -- start start 2024-04-23T18:28:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff184e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff1832b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff185a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1b26d0 con 0x555ff1837f40 2024-04-23T18:28:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1affa0 con 0x555ff1842800 2024-04-23T18:28:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed686b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ade10 con 0x555ff1837620 2024-04-23T18:28:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff1832b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff1832b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56950/0 (socket says 172.21.15.37:56950) 2024-04-23T18:28:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2036250911 learned_addr learned my addr 172.21.15.37:0/2036250911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff184e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed20ec640 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff185a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2036250911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 msgr2=0x555ff184e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff184e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2036250911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 msgr2=0x555ff185a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff185a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2036250911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff1844f60 con 0x555ff1837f40 2024-04-23T18:28:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed20ec640 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff185a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.342+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff1832b30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0096c0 tx=0x7f3ebc03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8868b76f045a3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3eb77fe640 1 -- 172.21.15.37:0/2036250911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc049070 con 0x555ff1837f40 2024-04-23T18:28:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3eb77fe640 1 -- 172.21.15.37:0/2036250911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ebc0c5750 con 0x555ff1837f40 2024-04-23T18:28:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3eb77fe640 1 -- 172.21.15.37:0/2036250911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0c52c0 con 0x555ff1837f40 2024-04-23T18:28:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3ed686b740 1 -- 172.21.15.37:0/2036250911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 msgr2=0x555ff1832b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff1832b30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0096c0 tx=0x7f3ebc03b460 comp rx=0 tx=0).stop 2024-04-23T18:28:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3ed686b740 1 -- 172.21.15.37:0/2036250911 shutdown_connections 2024-04-23T18:28:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.343+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1842800 0x555ff185a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff1832b30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2036250911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1837620 0x555ff184e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- 172.21.15.37:0/2036250911 >> 172.21.15.37:0/2036250911 conn(0x555ff1833b30 msgr2=0x555ff1832420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- 172.21.15.37:0/2036250911 shutdown_connections 2024-04-23T18:28:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- 172.21.15.37:0/2036250911 wait complete. 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- start start 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 Processor -- start 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837620 0x555ff1848170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff18486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1842800 0x555ff1848bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ae460 con 0x555ff1837f40 2024-04-23T18:28:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1af8d0 con 0x555ff1837620 2024-04-23T18:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1aeb30 con 0x555ff1842800 2024-04-23T18:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.344+0000 7f3ed686b740 10 client.0 _ll_register_callbacks cb 0x7f3ebc0f43b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff18486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed20ec640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1842800 0x555ff1848bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff18486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56952/0 (socket says 172.21.15.37:56952) 2024-04-23T18:28:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2501548482 learned_addr learned my addr 172.21.15.37:0/2501548482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837620 0x555ff1848170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2501548482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1842800 msgr2=0x555ff1848bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1842800 0x555ff1848bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837620 msgr2=0x555ff1848170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837620 0x555ff1848170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 -- 172.21.15.37:0/2501548482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff18b7d50 con 0x555ff1837f40 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed10ea640 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff18486b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0c5220 tx=0x7f3ebc0f0ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4786da3eaeaa1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2501548482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0f4030 con 0x555ff1837f40 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2501548482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f3ebc0c52c0 con 0x555ff1837f40 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ebc0c5750 con 0x555ff1837f40 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2501548482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc049050 con 0x555ff1837f40 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed686b740 10 client.16521 fetch_fsmap learned FSMap version 37 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.345+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555ff18b7d50 con 0x555ff1837f40 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2501548482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ebc0c5750 con 0x555ff1837f40 2024-04-23T18:28:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2501548482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 37) ==== 8342+0+0 (secure 0 0 0) 0x7f3ebc031ac0 con 0x555ff1837f40 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed686b740 10 client.16521 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed686b740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed686b740 20 client.16521 populate_metadata read hostname 'smithi037' 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff18b92d0 0x555ff18bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed686b740 4 client.16521 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ff183fcd0 con 0x555ff18b92d0 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff18b92d0 0x555ff18bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3ed18eb640 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff18b92d0 0x555ff18bb6c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693ff1aea529dc25 server_cookie=ede14538898db1f9 in_seq=0 out_seq=0 2024-04-23T18:28:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.346+0000 7f3eb57fa640 10 client.16521 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3eb57fa640 1 -- 172.21.15.37:0/2501548482 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ff183fcd0 con 0x555ff18b92d0 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3eb57fa640 10 client.16521 handle_command_reply: tid=0 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 client.16521 shutdown 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 msgr2=0x555ff18486b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff18486b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0c5220 tx=0x7f3ebc0f0ee0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 shutdown_connections 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555ff18b92d0 0x555ff18bb6c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff1842800 0x555ff1848bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ff1837f40 0x555ff18486b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 --2- 172.21.15.37:0/2501548482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ff1837620 0x555ff1848170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3eb57fa640 0 client.16521 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 >> 172.21.15.37:0/2501548482 conn(0x555ff1833b30 msgr2=0x555ff1832cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 shutdown_connections 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 1 -- 172.21.15.37:0/2501548482 wait complete. 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.379+0000 7f3ed686b740 20 client.16521 trim_cache size 0 max 16384 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:28:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:28:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:28:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:28:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f3eb80a3dc0 msgr2=0x7f3eb80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f3eb80a3dc0 0x7f3eb80a6280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4005c40 tx=0x7f3ec4004ab0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 msgr2=0x7f3ecc1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc1a0b60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005940 tx=0x7f3ebc0040c0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 shutdown_connections 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f3eb80a3dc0 0x7f3eb80a6280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ecc10e8b0 0x7f3ecc19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ecc1016b0 0x7f3ecc1a0b60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 --2- 172.21.15.37:0/1086524871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc100d00 0x7f3ecc1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 >> 172.21.15.37:0/1086524871 conn(0x7f3ecc077fd0 msgr2=0x7f3ecc10fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 shutdown_connections 2024-04-23T18:28:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:28:49.380+0000 7f3ed3b4f640 1 -- 172.21.15.37:0/1086524871 wait complete. 2024-04-23T18:28:49.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 419115 3.29 MB/sec execute 294 sec latency 42.465 ms 2024-04-23T18:28:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:50 smithi172 ceph-mon[32456]: pgmap v259: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-23T18:28:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:50 smithi173 ceph-mon[32906]: pgmap v259: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-23T18:28:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:50 smithi037 ceph-mon[29404]: pgmap v259: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-23T18:28:50.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 420275 3.30 MB/sec execute 295 sec latency 213.137 ms 2024-04-23T18:28:51.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 420275 3.29 MB/sec execute 296 sec latency 1213.218 ms 2024-04-23T18:28:52.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:52 smithi172 ceph-mon[32456]: pgmap v260: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-23T18:28:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:52 smithi173 ceph-mon[32906]: pgmap v260: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-23T18:28:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:52 smithi037 ceph-mon[29404]: pgmap v260: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-04-23T18:28:52.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 421000 3.28 MB/sec execute 297 sec latency 1946.707 ms 2024-04-23T18:28:53.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 422721 3.31 MB/sec execute 298 sec latency 11.998 ms 2024-04-23T18:28:54.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:54 smithi172 ceph-mon[32456]: pgmap v261: 97 pgs: 97 active+clean; 243 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-04-23T18:28:54.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:54 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:54.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:54 smithi173 ceph-mon[32906]: pgmap v261: 97 pgs: 97 active+clean; 243 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-04-23T18:28:54.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:54 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:54.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:54 smithi037 ceph-mon[29404]: pgmap v261: 97 pgs: 97 active+clean; 243 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-04-23T18:28:54.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:54 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:28:54.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 423824 3.32 MB/sec execute 299 sec latency 291.167 ms 2024-04-23T18:28:55.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 423824 3.31 MB/sec execute 300 sec latency 1291.254 ms 2024-04-23T18:28:56.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:56 smithi172 ceph-mon[32456]: pgmap v262: 97 pgs: 97 active+clean; 243 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:28:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:56 smithi173 ceph-mon[32906]: pgmap v262: 97 pgs: 97 active+clean; 243 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:28:56.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:56 smithi037 ceph-mon[29404]: pgmap v262: 97 pgs: 97 active+clean; 243 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:28:56.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 423824 3.30 MB/sec execute 301 sec latency 2291.315 ms 2024-04-23T18:28:57.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 423824 3.29 MB/sec execute 302 sec latency 3291.368 ms 2024-04-23T18:28:58.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:28:58 smithi172 ceph-mon[32456]: pgmap v263: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-04-23T18:28:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:28:58 smithi173 ceph-mon[32906]: pgmap v263: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-04-23T18:28:58.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:28:58 smithi037 ceph-mon[29404]: pgmap v263: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-04-23T18:28:58.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 424562 3.28 MB/sec execute 303 sec latency 4008.753 ms 2024-04-23T18:28:59.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 426230 3.31 MB/sec execute 304 sec latency 13.618 ms 2024-04-23T18:29:00.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:00 smithi172 ceph-mon[32456]: pgmap v264: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:29:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:00 smithi173 ceph-mon[32906]: pgmap v264: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:29:00.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:00 smithi037 ceph-mon[29404]: pgmap v264: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:29:00.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 427365 3.32 MB/sec execute 305 sec latency 236.591 ms 2024-04-23T18:29:01.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 427365 3.31 MB/sec execute 306 sec latency 1236.690 ms 2024-04-23T18:29:02.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:02 smithi172 ceph-mon[32456]: pgmap v265: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:29:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:02 smithi173 ceph-mon[32906]: pgmap v265: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:29:02.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:02 smithi037 ceph-mon[29404]: pgmap v265: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-04-23T18:29:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.478+0000 7efd6b719640 1 -- 172.21.15.37:0/162920922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 msgr2=0x7efd64112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.478+0000 7efd6b719640 1 --2- 172.21.15.37:0/162920922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd64112a20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efd54009900 tx=0x7efd5402a480 comp rx=0 tx=0).stop 2024-04-23T18:29:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 -- 172.21.15.37:0/162920922 shutdown_connections 2024-04-23T18:29:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 --2- 172.21.15.37:0/162920922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd64112f60 0x7efd64115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 --2- 172.21.15.37:0/162920922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd64112a20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 --2- 172.21.15.37:0/162920922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd640742f0 0x7efd640746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 -- 172.21.15.37:0/162920922 >> 172.21.15.37:0/162920922 conn(0x7efd640fda20 msgr2=0x7efd640ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 -- 172.21.15.37:0/162920922 shutdown_connections 2024-04-23T18:29:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 -- 172.21.15.37:0/162920922 wait complete. 2024-04-23T18:29:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 Processor -- start 2024-04-23T18:29:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.479+0000 7efd6b719640 1 -- start start 2024-04-23T18:29:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd6b719640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd640742f0 0x7efd641a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd6b719640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd641a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd6b719640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64112f60 0x7efd6419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd6b719640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd641187d0 con 0x7efd64074c10 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd6b719640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd64118650 con 0x7efd64112f60 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd6b719640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd64118950 con 0x7efd640742f0 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd641a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd641a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34112/0 (socket says 172.21.15.37:34112) 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 -- 172.21.15.37:0/2682016455 learned_addr learned my addr 172.21.15.37:0/2682016455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd69cb6640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64112f60 0x7efd6419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd694b5640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd640742f0 0x7efd641a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 -- 172.21.15.37:0/2682016455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd640742f0 msgr2=0x7efd641a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd640742f0 0x7efd641a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 -- 172.21.15.37:0/2682016455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64112f60 msgr2=0x7efd6419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64112f60 0x7efd6419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 -- 172.21.15.37:0/2682016455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6419f430 con 0x7efd64074c10 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd694b5640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd640742f0 0x7efd641a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd68cb4640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd641a4fa0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efd54005940 tx=0x7efd54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.480+0000 7efd5a7fc640 1 -- 172.21.15.37:0/2682016455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5403f030 con 0x7efd64074c10 2024-04-23T18:29:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.481+0000 7efd5a7fc640 1 -- 172.21.15.37:0/2682016455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54003aa0 con 0x7efd64074c10 2024-04-23T18:29:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.481+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6419f6c0 con 0x7efd64074c10 2024-04-23T18:29:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.481+0000 7efd5a7fc640 1 -- 172.21.15.37:0/2682016455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54035070 con 0x7efd64074c10 2024-04-23T18:29:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.481+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641ab9e0 con 0x7efd64074c10 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd5a7fc640 1 -- 172.21.15.37:0/2682016455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd54042030 con 0x7efd64074c10 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 Processor -- start 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 -- start start 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce8ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce98bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f53cdf0970 con 0x55f53ce8bf40 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f53cd8d4a0 con 0x55f53ce96800 2024-04-23T18:29:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd6e42b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f53cdec930 con 0x55f53ce8b620 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd694b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd694b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34114/0 (socket says 172.21.15.37:34114) 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.484+0000 7efd694b5640 1 -- 172.21.15.37:0/1312634197 learned_addr learned my addr 172.21.15.37:0/1312634197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd5a7fc640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7efd500a3e40 0x7efd500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd68cb4640 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce98bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd69cb6640 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce8ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd694b5640 1 -- 172.21.15.37:0/1312634197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 msgr2=0x55f53ce8ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd694b5640 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce8ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd69cb6640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7efd500a3e40 0x7efd500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd694b5640 1 -- 172.21.15.37:0/1312634197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 msgr2=0x55f53ce98bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd694b5640 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce98bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd694b5640 1 -- 172.21.15.37:0/1312634197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f53cb03210 con 0x55f53ce8bf40 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd5a7fc640 1 -- 172.21.15.37:0/2682016455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7efd540f0be0 con 0x7efd64074c10 2024-04-23T18:29:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd694b5640 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce962c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd60009d50 tx=0x7efd600117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a756114069b7b6fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd4f7fe640 1 -- 172.21.15.37:0/1312634197 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60012650 con 0x55f53ce8bf40 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd69cb6640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7efd500a3e40 0x7efd500a6300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009990 tx=0x7efd5c0096a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd4f7fe640 1 -- 172.21.15.37:0/1312634197 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd60009e40 con 0x55f53ce8bf40 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.485+0000 7efd4f7fe640 1 -- 172.21.15.37:0/1312634197 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6000fc00 con 0x55f53ce8bf40 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 -- 172.21.15.37:0/1312634197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 msgr2=0x55f53ce962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce962c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd60009d50 tx=0x7efd600117b0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 -- 172.21.15.37:0/1312634197 shutdown_connections 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce98bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce962c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1312634197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce8ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 -- 172.21.15.37:0/1312634197 >> 172.21.15.37:0/1312634197 conn(0x55f53ce87b30 msgr2=0x55f53ce87f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 -- 172.21.15.37:0/1312634197 shutdown_connections 2024-04-23T18:29:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 -- 172.21.15.37:0/1312634197 wait complete. 2024-04-23T18:29:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.486+0000 7efd6e42b740 1 -- start start 2024-04-23T18:29:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 Processor -- start 2024-04-23T18:29:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce96150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce93440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f53ce99430 con 0x55f53ce8bf40 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f53cd8b0b0 con 0x55f53ce96800 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f53ce995b0 con 0x55f53ce8b620 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd6e42b740 10 client.0 _ll_register_callbacks cb 0x55f53cd78d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd68cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce93440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd68cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce93440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34130/0 (socket says 172.21.15.37:34130) 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd68cb4640 1 -- 172.21.15.37:0/2226393309 learned_addr learned my addr 172.21.15.37:0/2226393309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd694b5640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce96150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.487+0000 7efd69cb6640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd68cb4640 1 -- 172.21.15.37:0/2226393309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 msgr2=0x55f53ce96150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd68cb4640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce96150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd68cb4640 1 -- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 msgr2=0x55f53ce939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd68cb4640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd68cb4640 1 -- 172.21.15.37:0/2226393309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53cbcf620 con 0x55f53ce8bf40 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd694b5640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce96150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd68cb4640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce93440 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd540b9510 tx=0x7efd5408b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b05934a49ffb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd4cff9640 1 -- 172.21.15.37:0/2226393309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54049070 con 0x55f53ce8bf40 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd4cff9640 1 -- 172.21.15.37:0/2226393309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7efd540b91b0 con 0x55f53ce8bf40 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f53cdfe390 con 0x55f53ce8bf40 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd4cff9640 1 -- 172.21.15.37:0/2226393309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efd540bc4a0 con 0x55f53ce8bf40 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd6e42b740 10 client.16533 fetch_fsmap learned FSMap version 38 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f53cea5360 con 0x55f53ce8bf40 2024-04-23T18:29:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.488+0000 7efd4cff9640 1 -- 172.21.15.37:0/2226393309 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f53cdfe390 con 0x55f53ce8bf40 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd4cff9640 1 -- 172.21.15.37:0/2226393309 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 8338+0+0 (secure 0 0 0) 0x7efd54003c50 con 0x55f53ce8bf40 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd6e42b740 10 client.16533 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd6e42b740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd6e42b740 20 client.16533 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd6e42b740 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cea67c0 0x55f53cea8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd6e42b740 4 client.16533 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f53cdf8200 con 0x55f53cea67c0 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd694b5640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cea67c0 0x55f53cea8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd4cff9640 10 client.16533 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.489+0000 7efd694b5640 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cea67c0 0x55f53cea8c10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50309c558e0e4a11 server_cookie=b189a6c342230250 in_seq=0 out_seq=0 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd4cff9640 1 -- 172.21.15.37:0/2226393309 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f53cdf8200 con 0x55f53cea67c0 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd4cff9640 10 client.16533 handle_command_reply: tid=0 2024-04-23T18:29:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 client.16533 shutdown 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 msgr2=0x55f53ce93440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce93440 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd540b9510 tx=0x7efd5408b4f0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 shutdown_connections 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cea67c0 0x55f53cea8c10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce93440 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd4cff9640 0 client.16533 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 --2- 172.21.15.37:0/2226393309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53ce96150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 >> 172.21.15.37:0/2226393309 conn(0x55f53ce87b30 msgr2=0x55f53ce8d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.490+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 shutdown_connections 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.491+0000 7efd6e42b740 1 -- 172.21.15.37:0/2226393309 wait complete. 2024-04-23T18:29:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.491+0000 7efd6e42b740 20 client.16533 trim_cache size 0 max 16384 2024-04-23T18:29:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 Processor -- start 2024-04-23T18:29:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 -- start start 2024-04-23T18:29:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53cea22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce87f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b6b60 con 0x55f53ce8bf40 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b4430 con 0x55f53ce96800 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd6e42b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b22a0 con 0x55f53ce8b620 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce87f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce87f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34132/0 (socket says 172.21.15.37:34132) 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 -- 172.21.15.37:0/672152082 learned_addr learned my addr 172.21.15.37:0/672152082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd694b5640 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53cea22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 -- 172.21.15.37:0/672152082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 msgr2=0x55f53cea22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53cea22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 -- 172.21.15.37:0/672152082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 msgr2=0x55f53ce95c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 -- 172.21.15.37:0/672152082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f53ce923f0 con 0x55f53ce8bf40 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.506+0000 7efd68cb4640 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce87f10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd540ee3a0 tx=0x7efd540dd050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65344c778440f119 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd4effd640 1 -- 172.21.15.37:0/672152082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54049070 con 0x55f53ce8bf40 2024-04-23T18:29:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd4effd640 1 -- 172.21.15.37:0/672152082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd540bd240 con 0x55f53ce8bf40 2024-04-23T18:29:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd4effd640 1 -- 172.21.15.37:0/672152082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5408e0d0 con 0x55f53ce8bf40 2024-04-23T18:29:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 -- 172.21.15.37:0/672152082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 msgr2=0x55f53ce87f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce87f10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd540ee3a0 tx=0x7efd540dd050 comp rx=0 tx=0).stop 2024-04-23T18:29:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 -- 172.21.15.37:0/672152082 shutdown_connections 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce96800 0x55f53ce95c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8bf40 0x55f53ce87f10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 --2- 172.21.15.37:0/672152082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce8b620 0x55f53cea22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 -- 172.21.15.37:0/672152082 >> 172.21.15.37:0/672152082 conn(0x55f53ce87b30 msgr2=0x55f53ce8eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 -- 172.21.15.37:0/672152082 shutdown_connections 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.507+0000 7efd6e42b740 1 -- 172.21.15.37:0/672152082 wait complete. 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 -- start start 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 Processor -- start 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 0x55f53ce9ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce8bf40 0x55f53ce99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce96800 0x55f53ce9a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b28f0 con 0x55f53ce8b620 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b3d60 con 0x55f53ce8bf40 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b2fc0 con 0x55f53ce96800 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd6e42b740 10 client.0 _ll_register_callbacks cb 0x7efd540a7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd694b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 0x55f53ce9ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd694b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 0x55f53ce9ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34140/0 (socket says 172.21.15.37:34140) 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.508+0000 7efd694b5640 1 -- 172.21.15.37:0/1478356083 learned_addr learned my addr 172.21.15.37:0/1478356083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd69cb6640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce96800 0x55f53ce9a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd68cb4640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce8bf40 0x55f53ce99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd694b5640 1 -- 172.21.15.37:0/1478356083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce96800 msgr2=0x55f53ce9a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd694b5640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce96800 0x55f53ce9a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd694b5640 1 -- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce8bf40 msgr2=0x55f53ce99e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd694b5640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce8bf40 0x55f53ce99e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd694b5640 1 -- 172.21.15.37:0/1478356083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53ce8bd70 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd68cb4640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce8bf40 0x55f53ce99e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd694b5640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 0x55f53ce9ea80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efd60009a60 tx=0x7efd60005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092569f0ca3bcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd4cff9640 1 -- 172.21.15.37:0/1478356083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60005540 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd4cff9640 1 -- 172.21.15.37:0/1478356083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7efd60029130 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd540bd240 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd4cff9640 1 -- 172.21.15.37:0/1478356083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efd60029330 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd6e42b740 10 client.16539 fetch_fsmap learned FSMap version 38 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.509+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f53cf0ca80 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd4cff9640 1 -- 172.21.15.37:0/1478356083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd540bd240 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd4cff9640 1 -- 172.21.15.37:0/1478356083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 8338+0+0 (secure 0 0 0) 0x7efd60003c90 con 0x55f53ce8b620 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd6e42b740 10 client.16539 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd6e42b740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd6e42b740 20 client.16539 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cf0ce70 0x55f53cf0f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd6e42b740 4 client.16539 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f53cd745f0 con 0x55f53cf0ce70 2024-04-23T18:29:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd68cb4640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cf0ce70 0x55f53cf0f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd4cff9640 10 client.16539 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.510+0000 7efd68cb4640 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cf0ce70 0x55f53cf0f260 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87f23ab84047039 server_cookie=87ee8bbe5c7c4b6 in_seq=0 out_seq=0 2024-04-23T18:29:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.550+0000 7efd4cff9640 1 -- 172.21.15.37:0/1478356083 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f53cd745f0 con 0x55f53cf0ce70 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.550+0000 7efd4cff9640 10 client.16539 handle_command_reply: tid=0 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.550+0000 7efd6e42b740 1 client.16539 shutdown 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.550+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 msgr2=0x55f53ce9ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.550+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 0x55f53ce9ea80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efd60009a60 tx=0x7efd60005040 comp rx=0 tx=0).stop 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 shutdown_connections 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55f53cf0ce70 0x55f53cf0f260 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f53ce96800 0x55f53ce9a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f53ce8bf40 0x55f53ce99e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd4cff9640 0 client.16539 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 --2- 172.21.15.37:0/1478356083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f53ce8b620 0x55f53ce9ea80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 >> 172.21.15.37:0/1478356083 conn(0x55f53ce87b30 msgr2=0x55f53ce86cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 shutdown_connections 2024-04-23T18:29:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 1 -- 172.21.15.37:0/1478356083 wait complete. 2024-04-23T18:29:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6e42b740 20 client.16539 trim_cache size 0 max 16384 2024-04-23T18:29:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:29:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:29:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:29:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7efd500a3e40 msgr2=0x7efd500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6b719640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7efd500a3e40 0x7efd500a6300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009990 tx=0x7efd5c0096a0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 msgr2=0x7efd641a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.551+0000 7efd6b719640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd641a4fa0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efd54005940 tx=0x7efd54004270 comp rx=0 tx=0).stop 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 shutdown_connections 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7efd500a3e40 0x7efd500a6300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64112f60 0x7efd6419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd64074c10 0x7efd641a4fa0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 --2- 172.21.15.37:0/2682016455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd640742f0 0x7efd641a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 >> 172.21.15.37:0/2682016455 conn(0x7efd640fda20 msgr2=0x7efd640ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 shutdown_connections 2024-04-23T18:29:02.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:02.552+0000 7efd6b719640 1 -- 172.21.15.37:0/2682016455 wait complete. 2024-04-23T18:29:02.837 INFO:tasks.workunit.client.0.smithi037.stdout: 1 427582 3.30 MB/sec execute 307 sec latency 1934.663 ms 2024-04-23T18:29:03.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 429359 3.32 MB/sec execute 308 sec latency 25.853 ms 2024-04-23T18:29:04.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:04 smithi172 ceph-mon[32456]: pgmap v266: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-04-23T18:29:04.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:04 smithi173 ceph-mon[32906]: pgmap v266: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-04-23T18:29:04.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:04 smithi037 ceph-mon[29404]: pgmap v266: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-04-23T18:29:04.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430502 3.33 MB/sec execute 309 sec latency 27.953 ms 2024-04-23T18:29:05.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:05 smithi172 ceph-mon[32456]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:29:05.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:05 smithi173 ceph-mon[32906]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:29:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:05 smithi037 ceph-mon[29404]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:29:05.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430914 3.32 MB/sec execute 310 sec latency 676.731 ms 2024-04-23T18:29:06.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:06 smithi172 ceph-mon[32456]: pgmap v268: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-04-23T18:29:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:06 smithi173 ceph-mon[32906]: pgmap v268: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-04-23T18:29:06.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:06 smithi037 ceph-mon[29404]: pgmap v268: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-04-23T18:29:06.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430914 3.31 MB/sec execute 311 sec latency 1676.778 ms 2024-04-23T18:29:07.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430914 3.30 MB/sec execute 312 sec latency 2676.827 ms 2024-04-23T18:29:08.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:08 smithi172 ceph-mon[32456]: pgmap v269: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:08.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:08 smithi173 ceph-mon[32906]: pgmap v269: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:08.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:08 smithi037 ceph-mon[29404]: pgmap v269: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:08.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 431651 3.30 MB/sec execute 313 sec latency 3393.508 ms 2024-04-23T18:29:09.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 433359 3.33 MB/sec execute 314 sec latency 14.222 ms 2024-04-23T18:29:10.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:10 smithi172 ceph-mon[32456]: pgmap v270: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:10.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:10 smithi173 ceph-mon[32906]: pgmap v270: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:10.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:10 smithi037 ceph-mon[29404]: pgmap v270: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:10.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 434458 3.33 MB/sec execute 315 sec latency 244.602 ms 2024-04-23T18:29:11.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 434458 3.32 MB/sec execute 316 sec latency 1244.673 ms 2024-04-23T18:29:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:11 smithi172 ceph-mon[32456]: pgmap v271: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:11 smithi172 ceph-mon[32456]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:29:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:11 smithi173 ceph-mon[32906]: pgmap v271: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:11 smithi173 ceph-mon[32906]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:29:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:11 smithi037 ceph-mon[29404]: pgmap v271: 97 pgs: 97 active+clean; 254 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-23T18:29:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:11 smithi037 ceph-mon[29404]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:29:12.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 435167 3.32 MB/sec execute 317 sec latency 1986.382 ms 2024-04-23T18:29:13.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 436882 3.34 MB/sec execute 318 sec latency 13.245 ms 2024-04-23T18:29:14.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:13 smithi172 ceph-mon[32456]: pgmap v273: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-04-23T18:29:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:13 smithi173 ceph-mon[32906]: pgmap v273: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-04-23T18:29:14.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:13 smithi037 ceph-mon[29404]: pgmap v273: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 212 op/s 2024-04-23T18:29:14.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 438005 3.35 MB/sec execute 319 sec latency 247.323 ms 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 -- 172.21.15.37:0/2082139215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40102b80 msgr2=0x7fee40106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 --2- 172.21.15.37:0/2082139215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40102b80 0x7fee40106bc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fee34009900 tx=0x7fee3402a410 comp rx=0 tx=0).stop 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 -- 172.21.15.37:0/2082139215 shutdown_connections 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 --2- 172.21.15.37:0/2082139215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 0x7fee4010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 --2- 172.21.15.37:0/2082139215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40102b80 0x7fee40106bc0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 --2- 172.21.15.37:0/2082139215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee401021d0 0x7fee401025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 -- 172.21.15.37:0/2082139215 >> 172.21.15.37:0/2082139215 conn(0x7fee400793e0 msgr2=0x7fee400797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 -- 172.21.15.37:0/2082139215 shutdown_connections 2024-04-23T18:29:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.653+0000 7fee44874640 1 -- 172.21.15.37:0/2082139215 wait complete. 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 Processor -- start 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 -- start start 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee401021d0 0x7fee40074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 0x7fee40074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 0x7fee401a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee401121c0 con 0x7fee40074cd0 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112040 con 0x7fee401072a0 2024-04-23T18:29:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.654+0000 7fee44874640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112340 con 0x7fee401021d0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 0x7fee401a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 0x7fee401a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51326/0 (socket says 172.21.15.37:51326) 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 -- 172.21.15.37:0/2299758987 learned_addr learned my addr 172.21.15.37:0/2299758987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3e59c640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee401021d0 0x7fee40074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 0x7fee40074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 -- 172.21.15.37:0/2299758987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee401021d0 msgr2=0x7fee40074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee401021d0 0x7fee40074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 -- 172.21.15.37:0/2299758987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 msgr2=0x7fee40074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 0x7fee40074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 -- 172.21.15.37:0/2299758987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee401a6f90 con 0x7fee40074cd0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 0x7fee40074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee3ed9d640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 0x7fee401a6890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fee3800cab0 tx=0x7fee3800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee2f7fe640 1 -- 172.21.15.37:0/2299758987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800adc0 con 0x7fee40074cd0 2024-04-23T18:29:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee2f7fe640 1 -- 172.21.15.37:0/2299758987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3800f900 con 0x7fee40074cd0 2024-04-23T18:29:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee401a7280 con 0x7fee40074cd0 2024-04-23T18:29:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee2f7fe640 1 -- 172.21.15.37:0/2299758987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380189a0 con 0x7fee40074cd0 2024-04-23T18:29:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.655+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a7750 con 0x7fee40074cd0 2024-04-23T18:29:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee2f7fe640 1 -- 172.21.15.37:0/2299758987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee3800f450 con 0x7fee40074cd0 2024-04-23T18:29:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 Processor -- start 2024-04-23T18:29:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 -- start start 2024-04-23T18:29:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf94a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94f40 0x55d76cf9f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cfa1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d76cef9970 con 0x55d76cf94620 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d76ce964a0 con 0x55d76cf94f40 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee47582740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d76cef5930 con 0x55d76cf9f800 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.658+0000 7fee2f7fe640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee280a3df0 0x7fee280a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf94a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf94a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51340/0 (socket says 172.21.15.37:51340) 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 -- 172.21.15.37:0/2036479417 learned_addr learned my addr 172.21.15.37:0/2036479417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3ed9d640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee280a3df0 0x7fee280a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3e59c640 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94f40 0x55d76cf9f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cfa1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee2f7fe640 1 -- 172.21.15.37:0/2299758987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fee380d0a90 con 0x7fee40074cd0 2024-04-23T18:29:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 -- 172.21.15.37:0/2036479417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 msgr2=0x55d76cfa1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cfa1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 -- 172.21.15.37:0/2036479417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94f40 msgr2=0x55d76cf9f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94f40 0x55d76cf9f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 -- 172.21.15.37:0/2036479417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76cc0c210 con 0x55d76cf94620 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3ed9d640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee280a3df0 0x7fee280a62b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee380ce7a0 tx=0x7fee3806f4e0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf94a00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee34005ab0 tx=0x7fee34009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40877ad1fb388fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee2cff9640 1 -- 172.21.15.37:0/2036479417 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34035050 con 0x55d76cf94620 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee2cff9640 1 -- 172.21.15.37:0/2036479417 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee340317a0 con 0x55d76cf94620 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.659+0000 7fee2cff9640 1 -- 172.21.15.37:0/2036479417 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee340046c0 con 0x55d76cf94620 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 -- 172.21.15.37:0/2036479417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 msgr2=0x55d76cf94a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf94a00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee34005ab0 tx=0x7fee34009560 comp rx=0 tx=0).stop 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 -- 172.21.15.37:0/2036479417 shutdown_connections 2024-04-23T18:29:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cfa1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94f40 0x55d76cf9f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 --2- 172.21.15.37:0/2036479417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf94a00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 -- 172.21.15.37:0/2036479417 >> 172.21.15.37:0/2036479417 conn(0x55d76cf90b30 msgr2=0x55d76cf90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 -- 172.21.15.37:0/2036479417 shutdown_connections 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.660+0000 7fee47582740 1 -- 172.21.15.37:0/2036479417 wait complete. 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 -- start start 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 Processor -- start 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf94f40 0x55d76cf9c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf9f800 0x55d76cf9c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d76cfa23d0 con 0x55d76cf94620 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d76ce940b0 con 0x55d76cf9f800 2024-04-23T18:29:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d76cfa2550 con 0x55d76cf94f40 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee47582740 10 client.0 _ll_register_callbacks cb 0x55d76ce81d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf9f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51354/0 (socket says 172.21.15.37:51354) 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 -- 172.21.15.37:0/1629988331 learned_addr learned my addr 172.21.15.37:0/1629988331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3e59c640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf94f40 0x55d76cf9c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 -- 172.21.15.37:0/1629988331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf94f40 msgr2=0x55d76cf9c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf94f40 0x55d76cf9c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 -- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf9f800 msgr2=0x55d76cf9c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf9f800 0x55d76cf9c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3dd9b640 1 -- 172.21.15.37:0/1629988331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76ccd8620 con 0x55d76cf94620 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.661+0000 7fee3e59c640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf94f40 0x55d76cf9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf9f140 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fee34005d50 tx=0x7fee340319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91244c0a07697950 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee25ffb640 1 -- 172.21.15.37:0/1629988331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34035040 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee25ffb640 1 -- 172.21.15.37:0/1629988331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fee340035f0 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee340317a0 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee25ffb640 1 -- 172.21.15.37:0/1629988331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee34057050 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 10 client.16551 fetch_fsmap learned FSMap version 38 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d76cfae2e0 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee25ffb640 1 -- 172.21.15.37:0/1629988331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee340317a0 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee25ffb640 1 -- 172.21.15.37:0/1629988331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 8338+0+0 (secure 0 0 0) 0x7fee34049640 con 0x55d76cf94620 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 10 client.16551 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 20 client.16551 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.662+0000 7fee47582740 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76cfaf630 0x55d76cfb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.663+0000 7fee47582740 4 client.16551 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.663+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d76cf07390 con 0x55d76cfaf630 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.663+0000 7fee3e59c640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76cfaf630 0x55d76cfb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.663+0000 7fee25ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.663+0000 7fee3e59c640 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76cfaf630 0x55d76cfb1a20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48148e67dad170f3 server_cookie=4d1ccd3d075ae054 in_seq=0 out_seq=0 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee25ffb640 1 -- 172.21.15.37:0/1629988331 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d76cf07390 con 0x55d76cfaf630 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee25ffb640 10 client.16551 handle_command_reply: tid=0 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 client.16551 shutdown 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 msgr2=0x55d76cf9f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf9f140 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fee34005d50 tx=0x7fee340319e0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 shutdown_connections 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76cfaf630 0x55d76cfb1a20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf9f800 0x55d76cf9c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf94f40 0x55d76cf9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee25ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 --2- 172.21.15.37:0/1629988331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94620 0x55d76cf9f140 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 >> 172.21.15.37:0/1629988331 conn(0x55d76cf90b30 msgr2=0x55d76cf968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 shutdown_connections 2024-04-23T18:29:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 1 -- 172.21.15.37:0/1629988331 wait complete. 2024-04-23T18:29:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.664+0000 7fee47582740 20 client.16551 trim_cache size 0 max 16384 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 Processor -- start 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 -- start start 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfab290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf97140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf9ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b2580 con 0x55d76cf94f40 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee401afe50 con 0x55d76cf94620 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee47582740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee401adce0 con 0x55d76cf9f800 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf97140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf97140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51356/0 (socket says 172.21.15.37:51356) 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 -- 172.21.15.37:0/995028497 learned_addr learned my addr 172.21.15.37:0/995028497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 -- 172.21.15.37:0/995028497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 msgr2=0x55d76cf9ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf9ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf9ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 -- 172.21.15.37:0/995028497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 msgr2=0x55d76cfab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 -- 172.21.15.37:0/995028497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76cf94d70 con 0x55d76cf94f40 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.680+0000 7fee3e59c640 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf97140 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fee300045c0 tx=0x7fee3000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be9258b9fa2ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee27fff640 1 -- 172.21.15.37:0/995028497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30023070 con 0x55d76cf94f40 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee27fff640 1 -- 172.21.15.37:0/995028497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee30003dc0 con 0x55d76cf94f40 2024-04-23T18:29:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee27fff640 1 -- 172.21.15.37:0/995028497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30015070 con 0x55d76cf94f40 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 -- 172.21.15.37:0/995028497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 msgr2=0x55d76cf97140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf97140 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fee300045c0 tx=0x7fee3000be40 comp rx=0 tx=0).stop 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 -- 172.21.15.37:0/995028497 shutdown_connections 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf9ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf97140 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 --2- 172.21.15.37:0/995028497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 -- 172.21.15.37:0/995028497 >> 172.21.15.37:0/995028497 conn(0x55d76cf90b30 msgr2=0x55d76cf96860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 -- 172.21.15.37:0/995028497 shutdown_connections 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.681+0000 7fee47582740 1 -- 172.21.15.37:0/995028497 wait complete. 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 -- start start 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 Processor -- start 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfa7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf98ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee401ae330 con 0x55d76cf94f40 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee401af780 con 0x55d76cf94620 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee401ae9e0 con 0x55d76cf9f800 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee47582740 10 client.0 _ll_register_callbacks cb 0x7fee34031550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf98ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf98ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51364/0 (socket says 172.21.15.37:51364) 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 -- 172.21.15.37:0/3200717318 learned_addr learned my addr 172.21.15.37:0/3200717318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfa7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 -- 172.21.15.37:0/3200717318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 msgr2=0x55d76cf995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 -- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 msgr2=0x55d76cfa7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfa7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3e59c640 1 -- 172.21.15.37:0/3200717318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76d0148e0 con 0x55d76cf94f40 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.682+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee3e59c640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf98ed0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fee3000b740 tx=0x7fee30004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e5e3c9560763b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee25ffb640 1 -- 172.21.15.37:0/3200717318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30015070 con 0x55d76cf94f40 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee25ffb640 1 -- 172.21.15.37:0/3200717318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fee30011c20 con 0x55d76cf94f40 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee30003dc0 con 0x55d76cf94f40 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee25ffb640 1 -- 172.21.15.37:0/3200717318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee30035050 con 0x55d76cf94f40 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 10 client.16557 fetch_fsmap learned FSMap version 38 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d76d015930 con 0x55d76cf94f40 2024-04-23T18:29:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee25ffb640 1 -- 172.21.15.37:0/3200717318 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee30003dc0 con 0x55d76cf94f40 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee25ffb640 1 -- 172.21.15.37:0/3200717318 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 38) ==== 8338+0+0 (secure 0 0 0) 0x7fee30030030 con 0x55d76cf94f40 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 10 client.16557 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 20 client.16557 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76d0160c0 0x55d76d0184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 4 client.16557 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.683+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d76cf01200 con 0x55d76d0160c0 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.684+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76d0160c0 0x55d76d0184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.684+0000 7fee3dd9b640 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76d0160c0 0x55d76d0184e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862f885b9c869222 server_cookie=c99d8f9ee80d4042 in_seq=0 out_seq=0 2024-04-23T18:29:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.684+0000 7fee25ffb640 10 client.16557 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee25ffb640 1 -- 172.21.15.37:0/3200717318 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d76cf01200 con 0x55d76d0160c0 2024-04-23T18:29:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee25ffb640 10 client.16557 handle_command_reply: tid=0 2024-04-23T18:29:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 client.16557 shutdown 2024-04-23T18:29:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 msgr2=0x55d76cf98ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf98ed0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fee3000b740 tx=0x7fee30004080 comp rx=0 tx=0).stop 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 shutdown_connections 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d76d0160c0 0x55d76d0184e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d76cf9f800 0x55d76cf995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d76cf94f40 0x55d76cf98ed0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee25ffb640 0 client.16557 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 --2- 172.21.15.37:0/3200717318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d76cf94620 0x55d76cfa7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.718+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 >> 172.21.15.37:0/3200717318 conn(0x55d76cf90b30 msgr2=0x55d76cf9bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 shutdown_connections 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee47582740 1 -- 172.21.15.37:0/3200717318 wait complete. 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee47582740 20 client.16557 trim_cache size 0 max 16384 2024-04-23T18:29:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:29:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:29:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:29:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:29:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee280a3df0 msgr2=0x7fee280a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee280a3df0 0x7fee280a62b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee380ce7a0 tx=0x7fee3806f4e0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 msgr2=0x7fee401a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 0x7fee401a6890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fee3800cab0 tx=0x7fee3800cf70 comp rx=0 tx=0).stop 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 shutdown_connections 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fee280a3df0 0x7fee280a62b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40074cd0 0x7fee401a6890 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee401072a0 0x7fee40074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 --2- 172.21.15.37:0/2299758987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee401021d0 0x7fee40074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 >> 172.21.15.37:0/2299758987 conn(0x7fee400793e0 msgr2=0x7fee400ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 shutdown_connections 2024-04-23T18:29:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:15.719+0000 7fee44874640 1 -- 172.21.15.37:0/2299758987 wait complete. 2024-04-23T18:29:15.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 438404 3.34 MB/sec execute 320 sec latency 1114.753 ms 2024-04-23T18:29:16.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:15 smithi172 ceph-mon[32456]: pgmap v274: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-04-23T18:29:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:15 smithi173 ceph-mon[32906]: pgmap v274: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-04-23T18:29:16.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:15 smithi037 ceph-mon[29404]: pgmap v274: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-04-23T18:29:16.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440238 3.37 MB/sec execute 321 sec latency 69.532 ms 2024-04-23T18:29:17.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 441547 3.38 MB/sec execute 322 sec latency 142.753 ms 2024-04-23T18:29:18.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:17 smithi172 ceph-mon[32456]: pgmap v275: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:17 smithi173 ceph-mon[32906]: pgmap v275: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:18.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:17 smithi037 ceph-mon[29404]: pgmap v275: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:18.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 442250 3.37 MB/sec execute 323 sec latency 857.437 ms 2024-04-23T18:29:19.838 INFO:tasks.workunit.client.0.smithi037.stdout: 1 443811 3.40 MB/sec execute 324 sec latency 13.313 ms 2024-04-23T18:29:20.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:19 smithi172 ceph-mon[32456]: pgmap v276: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:19 smithi173 ceph-mon[32906]: pgmap v276: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:20.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:19 smithi037 ceph-mon[29404]: pgmap v276: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:20.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 445096 3.41 MB/sec execute 325 sec latency 36.572 ms 2024-04-23T18:29:21.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 445096 3.40 MB/sec execute 326 sec latency 1036.662 ms 2024-04-23T18:29:22.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:21 smithi172 ceph-mon[32456]: pgmap v277: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:21 smithi173 ceph-mon[32906]: pgmap v277: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:22.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:21 smithi037 ceph-mon[29404]: pgmap v277: 97 pgs: 97 active+clean; 260 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-04-23T18:29:22.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 445789 3.39 MB/sec execute 327 sec latency 1776.031 ms 2024-04-23T18:29:23.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447322 3.42 MB/sec execute 328 sec latency 39.036 ms 2024-04-23T18:29:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:23 smithi172 ceph-mon[32456]: pgmap v278: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-04-23T18:29:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:23 smithi173 ceph-mon[32906]: pgmap v278: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-04-23T18:29:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:23 smithi037 ceph-mon[29404]: pgmap v278: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-04-23T18:29:24.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447551 3.41 MB/sec execute 329 sec latency 606.861 ms 2024-04-23T18:29:25.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:24 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:29:25.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:24 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:29:25.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:24 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:29:25.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447551 3.40 MB/sec execute 330 sec latency 1606.974 ms 2024-04-23T18:29:26.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:26 smithi172 ceph-mon[32456]: pgmap v279: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-04-23T18:29:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:26 smithi173 ceph-mon[32906]: pgmap v279: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-04-23T18:29:26.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:26 smithi037 ceph-mon[29404]: pgmap v279: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-04-23T18:29:26.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447551 3.39 MB/sec execute 331 sec latency 2607.086 ms 2024-04-23T18:29:27.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447551 3.38 MB/sec execute 332 sec latency 3607.139 ms 2024-04-23T18:29:28.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:28 smithi172 ceph-mon[32456]: pgmap v280: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-04-23T18:29:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:28 smithi173 ceph-mon[32906]: pgmap v280: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-04-23T18:29:28.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:28 smithi037 ceph-mon[29404]: pgmap v280: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-04-23T18:29:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 -- 172.21.15.37:0/3985736946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa574107070 msgr2=0x7fa574107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 --2- 172.21.15.37:0/3985736946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa574107070 0x7fa574107450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa564009930 tx=0x7fa56402a420 comp rx=0 tx=0).stop 2024-04-23T18:29:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 -- 172.21.15.37:0/3985736946 shutdown_connections 2024-04-23T18:29:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 --2- 172.21.15.37:0/3985736946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5741009e0 0x7fa57410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 --2- 172.21.15.37:0/3985736946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574100020 0x7fa5741004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 --2- 172.21.15.37:0/3985736946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa574107070 0x7fa574107450 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 -- 172.21.15.37:0/3985736946 >> 172.21.15.37:0/3985736946 conn(0x7fa574076f50 msgr2=0x7fa5740fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 -- 172.21.15.37:0/3985736946 shutdown_connections 2024-04-23T18:29:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.817+0000 7fa578e72640 1 -- 172.21.15.37:0/3985736946 wait complete. 2024-04-23T18:29:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.818+0000 7fa578e72640 1 Processor -- start 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.818+0000 7fa578e72640 1 -- start start 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa578e72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa574100020 0x7fa5741a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa578e72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 0x7fa5741a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa578e72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574107070 0x7fa57419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa578e72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741121b0 con 0x7fa5741009e0 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa578e72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa574112030 con 0x7fa574107070 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa578e72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa574112330 con 0x7fa574100020 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 0x7fa5741a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 0x7fa5741a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52960/0 (socket says 172.21.15.37:52960) 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 -- 172.21.15.37:0/1235732684 learned_addr learned my addr 172.21.15.37:0/1235732684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa572d9d640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574107070 0x7fa57419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa57259c640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa574100020 0x7fa5741a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 -- 172.21.15.37:0/1235732684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa574100020 msgr2=0x7fa5741a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa574100020 0x7fa5741a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 -- 172.21.15.37:0/1235732684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574107070 msgr2=0x7fa57419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574107070 0x7fa57419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa571d9b640 1 -- 172.21.15.37:0/1235732684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57419af90 con 0x7fa5741009e0 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa572d9d640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574107070 0x7fa57419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.819+0000 7fa57259c640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa574100020 0x7fa5741a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.820+0000 7fa571d9b640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 0x7fa5741a0b50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9f0 tx=0x7fa56800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.820+0000 7fa5637fe640 1 -- 172.21.15.37:0/1235732684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56800cd80 con 0x7fa5741009e0 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.820+0000 7fa5637fe640 1 -- 172.21.15.37:0/1235732684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5680108e0 con 0x7fa5741009e0 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.820+0000 7fa5637fe640 1 -- 172.21.15.37:0/1235732684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568018980 con 0x7fa5741009e0 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.820+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57419b280 con 0x7fa5741009e0 2024-04-23T18:29:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.820+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5741a7590 con 0x7fa5741009e0 2024-04-23T18:29:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa5637fe640 1 -- 172.21.15.37:0/1235732684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa568010430 con 0x7fa5741009e0 2024-04-23T18:29:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 Processor -- start 2024-04-23T18:29:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 -- start start 2024-04-23T18:29:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa5637fe640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa55c0a3e40 0x7fa55c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559958432970 con 0x5599584d8800 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5599583cf4a0 con 0x5599584cd620 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.823+0000 7fa57bb83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55995842e930 con 0x5599584cdf40 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa5637fe640 1 -- 172.21.15.37:0/1235732684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fa5680d0ab0 con 0x7fa5741009e0 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa572d9d640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa55c0a3e40 0x7fa55c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:36150/0 (socket says 172.21.15.37:36150) 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 -- 172.21.15.37:0/2058197375 learned_addr learned my addr 172.21.15.37:0/2058197375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa571d9b640 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa572d9d640 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 -- 172.21.15.37:0/2058197375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 msgr2=0x5599584d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584d82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 -- 172.21.15.37:0/2058197375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 msgr2=0x5599584dabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584dabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 -- 172.21.15.37:0/2058197375 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559958145210 con 0x5599584cd620 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa571d9b640 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa572d9d640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa55c0a3e40 0x7fa55c0a6300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa56c005fd0 tx=0x7fa56c011720 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa57259c640 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584cda00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa564005ec0 tx=0x7fa564032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4d9f649101d671 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa560ff9640 1 -- 172.21.15.37:0/2058197375 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564035040 con 0x5599584cd620 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa560ff9640 1 -- 172.21.15.37:0/2058197375 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa564031720 con 0x5599584cd620 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.824+0000 7fa560ff9640 1 -- 172.21.15.37:0/2058197375 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564003d80 con 0x5599584cd620 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 -- 172.21.15.37:0/2058197375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 msgr2=0x5599584cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584cda00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa564005ec0 tx=0x7fa564032e90 comp rx=0 tx=0).stop 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 -- 172.21.15.37:0/2058197375 shutdown_connections 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 --2- 172.21.15.37:0/2058197375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584cda00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 -- 172.21.15.37:0/2058197375 >> 172.21.15.37:0/2058197375 conn(0x5599584c9b30 msgr2=0x5599584c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 -- 172.21.15.37:0/2058197375 shutdown_connections 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.825+0000 7fa57bb83740 1 -- 172.21.15.37:0/2058197375 wait complete. 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 -- start start 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 Processor -- start 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 0x5599584d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cdf40 0x5599584d5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599584db3d0 con 0x5599584d8800 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5599583cd0b0 con 0x5599584cdf40 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5599584db550 con 0x5599584cd620 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57bb83740 10 client.0 _ll_register_callbacks cb 0x5599583bad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 0x5599584d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 0x5599584d8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:48318/0 (socket says 172.21.15.37:48318) 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 -- 172.21.15.37:0/1932348771 learned_addr learned my addr 172.21.15.37:0/1932348771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa571d9b640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cdf40 0x5599584d5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 -- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cdf40 msgr2=0x5599584d5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cdf40 0x5599584d5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 -- 172.21.15.37:0/1932348771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 msgr2=0x5599584d58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584d58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa57259c640 1 -- 172.21.15.37:0/1932348771 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559958211620 con 0x5599584cd620 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.826+0000 7fa572d9d640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa57259c640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 0x5599584d8090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa56402a940 tx=0x7fa564005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81b5efb29841f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa559ffb640 1 -- 172.21.15.37:0/1932348771 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56404e070 con 0x5599584cd620 2024-04-23T18:29:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa564031720 con 0x5599584cd620 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa559ffb640 1 -- 172.21.15.37:0/1932348771 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fa564003d30 con 0x5599584cd620 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa559ffb640 1 -- 172.21.15.37:0/1932348771 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa56405a050 con 0x5599584cd620 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa57bb83740 10 client.26032 fetch_fsmap learned FSMap version 39 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5599584db3d0 con 0x5599584cd620 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa559ffb640 1 -- 172.21.15.37:0/1932348771 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa564031720 con 0x5599584cd620 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.827+0000 7fa559ffb640 1 -- 172.21.15.37:0/1932348771 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 39) ==== 8342+0+0 (secure 0 0 0) 0x7fa56403c960 con 0x5599584cd620 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa57bb83740 10 client.26032 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa57bb83740 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa57bb83740 20 client.26032 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa57bb83740 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5599584e83d0 0x5599584ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa57bb83740 4 client.26032 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559958440390 con 0x5599584e83d0 2024-04-23T18:29:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa571d9b640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5599584e83d0 0x5599584ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa559ffb640 10 client.26032 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.828+0000 7fa571d9b640 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5599584e83d0 0x5599584ea7c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15615a96ff7fcc3b server_cookie=c9a46d3e6d71eee4 in_seq=0 out_seq=0 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa559ffb640 1 -- 172.21.15.37:0/1932348771 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559958440390 con 0x5599584e83d0 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa559ffb640 10 client.26032 handle_command_reply: tid=0 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 client.26032 shutdown 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 msgr2=0x5599584d8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 0x5599584d8090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa56402a940 tx=0x7fa564005fd0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 shutdown_connections 2024-04-23T18:29:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5599584e83d0 0x5599584ea7c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cdf40 0x5599584d5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 --2- 172.21.15.37:0/1932348771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cd620 0x5599584d8090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa559ffb640 0 client.26032 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 >> 172.21.15.37:0/1932348771 conn(0x5599584c9b30 msgr2=0x5599584cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 shutdown_connections 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 1 -- 172.21.15.37:0/1932348771 wait complete. 2024-04-23T18:29:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.829+0000 7fa57bb83740 20 client.26032 trim_cache size 0 max 16384 2024-04-23T18:29:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 Processor -- start 2024-04-23T18:29:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 -- start start 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 0x5599584d7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584d8800 0x5599584f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741b26c0 con 0x5599584cd620 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741aff60 con 0x5599584d8800 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.846+0000 7fa57bb83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741addd0 con 0x5599584cdf40 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 0x5599584d7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 0x5599584d7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52984/0 (socket says 172.21.15.37:52984) 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 -- 172.21.15.37:0/3087685304 learned_addr learned my addr 172.21.15.37:0/3087685304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584d8800 0x5599584f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa571d9b640 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 -- 172.21.15.37:0/3087685304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 msgr2=0x5599584eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 -- 172.21.15.37:0/3087685304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584d8800 msgr2=0x5599584f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584d8800 0x5599584f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 -- 172.21.15.37:0/3087685304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599584d5ec0 con 0x5599584cd620 2024-04-23T18:29:28.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 447996 3.37 MB/sec execute 333 sec latency 4320.963 ms 2024-04-23T18:29:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa57259c640 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 0x5599584d7bc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa564005c40 tx=0x7fa564003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ef00d48e86b9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa55bfff640 1 -- 172.21.15.37:0/3087685304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56404e070 con 0x5599584cd620 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa55bfff640 1 -- 172.21.15.37:0/3087685304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa564004000 con 0x5599584cd620 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.847+0000 7fa55bfff640 1 -- 172.21.15.37:0/3087685304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564035040 con 0x5599584cd620 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 -- 172.21.15.37:0/3087685304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 msgr2=0x5599584d7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 0x5599584d7bc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa564005c40 tx=0x7fa564003ab0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 -- 172.21.15.37:0/3087685304 shutdown_connections 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584d8800 0x5599584f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 --2- 172.21.15.37:0/3087685304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584cd620 0x5599584d7bc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 -- 172.21.15.37:0/3087685304 >> 172.21.15.37:0/3087685304 conn(0x5599584c9b30 msgr2=0x5599584cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 -- 172.21.15.37:0/3087685304 shutdown_connections 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 -- 172.21.15.37:0/3087685304 wait complete. 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 -- start start 2024-04-23T18:29:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.848+0000 7fa57bb83740 1 Processor -- start 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741ae420 con 0x5599584d8800 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741af890 con 0x5599584cd620 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5741aeaf0 con 0x5599584cdf40 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57bb83740 10 client.0 _ll_register_callbacks cb 0x7fa568098640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa572d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584e07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:36166/0 (socket says 172.21.15.37:36166) 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57259c640 1 -- 172.21.15.37:0/4173350636 learned_addr learned my addr 172.21.15.37:0/4173350636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa571d9b640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa572d9d640 1 -- 172.21.15.37:0/4173350636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 msgr2=0x5599584e0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa572d9d640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584e0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa572d9d640 1 -- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 msgr2=0x5599584e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa572d9d640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa572d9d640 1 -- 172.21.15.37:0/4173350636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599584ead20 con 0x5599584d8800 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa571d9b640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584e0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.849+0000 7fa57259c640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa572d9d640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584ddd40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ffd0 tx=0x7fa56c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14fa9de84f7bf7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa559ffb640 1 -- 172.21.15.37:0/4173350636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c026040 con 0x5599584d8800 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa559ffb640 1 -- 172.21.15.37:0/4173350636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fa56c01a070 con 0x5599584d8800 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa564004000 con 0x5599584d8800 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa559ffb640 1 -- 172.21.15.37:0/4173350636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa56c037050 con 0x5599584d8800 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa57bb83740 10 client.16575 fetch_fsmap learned FSMap version 39 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.850+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55995854e260 con 0x5599584d8800 2024-04-23T18:29:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa559ffb640 1 -- 172.21.15.37:0/4173350636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c011410 con 0x5599584d8800 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa559ffb640 1 -- 172.21.15.37:0/4173350636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8342+0+0 (secure 0 0 0) 0x7fa56c029070 con 0x5599584d8800 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57bb83740 10 client.16575 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57bb83740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57bb83740 20 client.16575 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57bb83740 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55995854ea30 0x559958550e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57bb83740 4 client.16575 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa564004000 con 0x55995854ea30 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.851+0000 7fa57259c640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55995854ea30 0x559958550e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.852+0000 7fa57259c640 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55995854ea30 0x559958550e50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7ed2fa009f4c39 server_cookie=f2f3e8d40aa994a0 in_seq=0 out_seq=0 2024-04-23T18:29:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.852+0000 7fa559ffb640 10 client.16575 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.882+0000 7fa559ffb640 1 -- 172.21.15.37:0/4173350636 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa564004000 con 0x55995854ea30 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.882+0000 7fa559ffb640 10 client.16575 handle_command_reply: tid=0 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.882+0000 7fa57bb83740 1 client.16575 shutdown 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 msgr2=0x5599584ddd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584ddd40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ffd0 tx=0x7fa56c009500 comp rx=0 tx=0).stop 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 shutdown_connections 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55995854ea30 0x559958550e50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599584d8800 0x5599584ddd40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5599584cdf40 0x5599584e0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa559ffb640 0 client.16575 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 --2- 172.21.15.37:0/4173350636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599584cd620 0x5599584e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 >> 172.21.15.37:0/4173350636 conn(0x5599584c9b30 msgr2=0x5599584c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 shutdown_connections 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 1 -- 172.21.15.37:0/4173350636 wait complete. 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa57bb83740 20 client.16575 trim_cache size 0 max 16384 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:29:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:29:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa55c0a3e40 msgr2=0x7fa55c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa55c0a3e40 0x7fa55c0a6300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa56c005fd0 tx=0x7fa56c011720 comp rx=0 tx=0).stop 2024-04-23T18:29:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 msgr2=0x7fa5741a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 0x7fa5741a0b50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9f0 tx=0x7fa56800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 shutdown_connections 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa55c0a3e40 0x7fa55c0a6300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa574107070 0x7fa57419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5741009e0 0x7fa5741a0b50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 --2- 172.21.15.37:0/1235732684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa574100020 0x7fa5741a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.883+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 >> 172.21.15.37:0/1235732684 conn(0x7fa574076f50 msgr2=0x7fa5740fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.884+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 shutdown_connections 2024-04-23T18:29:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:28.884+0000 7fa578e72640 1 -- 172.21.15.37:0/1235732684 wait complete. 2024-04-23T18:29:29.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 448638 3.37 MB/sec execute 334 sec latency 629.007 ms 2024-04-23T18:29:30.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:30 smithi172 ceph-mon[32456]: pgmap v281: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-23T18:29:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:30 smithi173 ceph-mon[32906]: pgmap v281: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-23T18:29:30.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:30 smithi037 ceph-mon[29404]: pgmap v281: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-23T18:29:30.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 448638 3.36 MB/sec execute 335 sec latency 1629.126 ms 2024-04-23T18:29:31.839 INFO:tasks.workunit.client.0.smithi037.stdout: 1 448638 3.35 MB/sec execute 336 sec latency 2629.237 ms 2024-04-23T18:29:32.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:32 smithi172 ceph-mon[32456]: pgmap v282: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-23T18:29:32.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:32 smithi173 ceph-mon[32906]: pgmap v282: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-23T18:29:32.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:32 smithi037 ceph-mon[29404]: pgmap v282: 97 pgs: 97 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-04-23T18:29:32.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 449336 3.35 MB/sec execute 337 sec latency 3367.879 ms 2024-04-23T18:29:33.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 451051 3.37 MB/sec execute 338 sec latency 14.008 ms 2024-04-23T18:29:34.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:34 smithi172 ceph-mon[32456]: pgmap v283: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-04-23T18:29:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:34 smithi173 ceph-mon[32906]: pgmap v283: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-04-23T18:29:34.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:34 smithi037 ceph-mon[29404]: pgmap v283: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-04-23T18:29:34.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 452187 3.38 MB/sec execute 339 sec latency 365.218 ms 2024-04-23T18:29:35.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 452187 3.37 MB/sec execute 340 sec latency 1365.276 ms 2024-04-23T18:29:36.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:36 smithi172 ceph-mon[32456]: pgmap v284: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1002 KiB/s wr, 82 op/s 2024-04-23T18:29:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:36 smithi173 ceph-mon[32906]: pgmap v284: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1002 KiB/s wr, 82 op/s 2024-04-23T18:29:36.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:36 smithi037 ceph-mon[29404]: pgmap v284: 97 pgs: 97 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1002 KiB/s wr, 82 op/s 2024-04-23T18:29:36.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 452187 3.36 MB/sec execute 341 sec latency 2365.372 ms 2024-04-23T18:29:37.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 452187 3.35 MB/sec execute 342 sec latency 3365.473 ms 2024-04-23T18:29:38.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:38 smithi172 ceph-mon[32456]: pgmap v285: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-04-23T18:29:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:38 smithi173 ceph-mon[32906]: pgmap v285: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-04-23T18:29:38.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:38 smithi037 ceph-mon[29404]: pgmap v285: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-04-23T18:29:38.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 452924 3.34 MB/sec execute 343 sec latency 4077.710 ms 2024-04-23T18:29:39.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 454479 3.37 MB/sec execute 344 sec latency 13.599 ms 2024-04-23T18:29:40.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:40 smithi172 ceph-mon[32456]: pgmap v286: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:29:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:40 smithi173 ceph-mon[32906]: pgmap v286: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:29:40.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:40 smithi037 ceph-mon[29404]: pgmap v286: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:29:40.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 455326 3.37 MB/sec execute 345 sec latency 20.303 ms 2024-04-23T18:29:41.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 455730 3.37 MB/sec execute 346 sec latency 548.438 ms 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.983+0000 7f7f3067c640 1 -- 172.21.15.37:0/3967363153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f281011a0 msgr2=0x7f7f28101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.983+0000 7f7f3067c640 1 --2- 172.21.15.37:0/3967363153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f281011a0 0x7f7f28101600 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009930 tx=0x7f7f1802a420 comp rx=0 tx=0).stop 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- 172.21.15.37:0/3967363153 shutdown_connections 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 --2- 172.21.15.37:0/3967363153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f28101b40 0x7f7f2810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 --2- 172.21.15.37:0/3967363153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f281011a0 0x7f7f28101600 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 --2- 172.21.15.37:0/3967363153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28106fe0 0x7f7f281073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- 172.21.15.37:0/3967363153 >> 172.21.15.37:0/3967363153 conn(0x7f7f28076b80 msgr2=0x7f7f28076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- 172.21.15.37:0/3967363153 shutdown_connections 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- 172.21.15.37:0/3967363153 wait complete. 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 Processor -- start 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- start start 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f281011a0 0x7f7f281986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 0x7f7f28198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28106fe0 0x7f7f2819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28112250 con 0x7f7f28101b40 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281120d0 con 0x7f7f28106fe0 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.984+0000 7f7f3067c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281123d0 con 0x7f7f281011a0 2024-04-23T18:29:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 0x7f7f28198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 0x7f7f28198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59170/0 (socket says 172.21.15.37:59170) 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 -- 172.21.15.37:0/1563797898 learned_addr learned my addr 172.21.15.37:0/1563797898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28106fe0 0x7f7f2819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2e418640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f281011a0 0x7f7f281986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 -- 172.21.15.37:0/1563797898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f281011a0 msgr2=0x7f7f281986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f281011a0 0x7f7f281986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 -- 172.21.15.37:0/1563797898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28106fe0 msgr2=0x7f7f2819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28106fe0 0x7f7f2819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 -- 172.21.15.37:0/1563797898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2819d6c0 con 0x7f7f28101b40 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2e418640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f281011a0 0x7f7f281986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 0x7f7f28198c10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7f18004770 tx=0x7f7f180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f1f7fe640 1 -- 172.21.15.37:0/1563797898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1803f070 con 0x7f7f28101b40 2024-04-23T18:29:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f1f7fe640 1 -- 172.21.15.37:0/1563797898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1803b460 con 0x7f7f28101b40 2024-04-23T18:29:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2819d950 con 0x7f7f28101b40 2024-04-23T18:29:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f1f7fe640 1 -- 172.21.15.37:0/1563797898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18035070 con 0x7f7f28101b40 2024-04-23T18:29:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.985+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281a5240 con 0x7f7f28101b40 2024-04-23T18:29:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f1f7fe640 1 -- 172.21.15.37:0/1563797898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7f1803b8d0 con 0x7f7f28101b40 2024-04-23T18:29:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 Processor -- start 2024-04-23T18:29:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 -- start start 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e46a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46f40 0x55c724e512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e51800 0x55c724e53bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c724dab970 con 0x55c724e46620 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c724d484a0 con 0x55c724e51800 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.988+0000 7f7f33398740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c724da7930 con 0x55c724e46f40 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e46a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e46a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59174/0 (socket says 172.21.15.37:59174) 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 -- 172.21.15.37:0/594105109 learned_addr learned my addr 172.21.15.37:0/594105109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f1f7fe640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f140a3e40 0x7f7f140a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e51800 0x55c724e53bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 -- 172.21.15.37:0/594105109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46f40 msgr2=0x55c724e512c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46f40 0x55c724e512c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 -- 172.21.15.37:0/594105109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e51800 msgr2=0x55c724e53bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e51800 0x55c724e53bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 -- 172.21.15.37:0/594105109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c724abe210 con 0x55c724e46620 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e51800 0x55c724e53bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f1f7fe640 1 -- 172.21.15.37:0/1563797898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f7f180f0bc0 con 0x7f7f28101b40 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f140a3e40 0x7f7f140a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e46a00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c4a0 tx=0x7f7f2000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369a41c0383d3e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f1cff9640 1 -- 172.21.15.37:0/594105109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20018640 con 0x55c724e46620 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.989+0000 7f7f1cff9640 1 -- 172.21.15.37:0/594105109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f2001e070 con 0x55c724e46620 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f1cff9640 1 -- 172.21.15.37:0/594105109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000a900 con 0x55c724e46620 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 -- 172.21.15.37:0/594105109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 msgr2=0x55c724e46a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f140a3e40 0x7f7f140a6300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f2001d000 tx=0x7f7f20017000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e46a00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c4a0 tx=0x7f7f2000c960 comp rx=0 tx=0).stop 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 -- 172.21.15.37:0/594105109 shutdown_connections 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e51800 0x55c724e53bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46f40 0x55c724e512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 --2- 172.21.15.37:0/594105109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e46a00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 -- 172.21.15.37:0/594105109 >> 172.21.15.37:0/594105109 conn(0x55c724e42b30 msgr2=0x55c724e42f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 -- 172.21.15.37:0/594105109 shutdown_connections 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.990+0000 7f7f33398740 1 -- 172.21.15.37:0/594105109 wait complete. 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 -- start start 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 Processor -- start 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46620 0x55c724e51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 0x55c724e4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c724e54430 con 0x55c724e51800 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c724d460b0 con 0x55c724e46f40 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c724e545b0 con 0x55c724e46620 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f33398740 10 client.0 _ll_register_callbacks cb 0x55c724d33d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f2ec19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 0x55c724e4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f2ec19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 0x55c724e4e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59178/0 (socket says 172.21.15.37:59178) 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f2ec19640 1 -- 172.21.15.37:0/3849256405 learned_addr learned my addr 172.21.15.37:0/3849256405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f2e418640 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46620 0x55c724e51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.991+0000 7f7f2ec19640 1 -- 172.21.15.37:0/3849256405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46620 msgr2=0x55c724e51140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46620 0x55c724e51140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f2ec19640 1 -- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 msgr2=0x55c724e4e430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4e430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f2ec19640 1 -- 172.21.15.37:0/3849256405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c724b8a620 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 0x55c724e4e970 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000a440 tx=0x7f7f2000c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ffd11b59f4902d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f11ffb640 1 -- 172.21.15.37:0/3849256405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000a6c0 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f11ffb640 1 -- 172.21.15.37:0/3849256405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f7f20018650 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f2001e070 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f11ffb640 1 -- 172.21.15.37:0/3849256405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f20018850 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f33398740 10 client.16587 fetch_fsmap learned FSMap version 39 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c724e60340 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f11ffb640 1 -- 172.21.15.37:0/3849256405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f2001e070 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f11ffb640 1 -- 172.21.15.37:0/3849256405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8342+0+0 (secure 0 0 0) 0x7f7f20032030 con 0x55c724e51800 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.992+0000 7f7f33398740 10 client.16587 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f33398740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f33398740 20 client.16587 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f33398740 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724e61740 0x55c724e63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f33398740 4 client.16587 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c724db9390 con 0x55c724e61740 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f2e418640 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724e61740 0x55c724e63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f2e418640 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724e61740 0x55c724e63b30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32473c4afc72858d server_cookie=4c4091ca1545d734 in_seq=0 out_seq=0 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.993+0000 7f7f11ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f11ffb640 1 -- 172.21.15.37:0/3849256405 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c724db9390 con 0x55c724e61740 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f11ffb640 10 client.16587 handle_command_reply: tid=0 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 client.16587 shutdown 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 msgr2=0x55c724e4e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 0x55c724e4e970 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000a440 tx=0x7f7f2000c0a0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 shutdown_connections 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724e61740 0x55c724e63b30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e51800 0x55c724e4e970 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 --2- 172.21.15.37:0/3849256405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e46620 0x55c724e51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f11ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 >> 172.21.15.37:0/3849256405 conn(0x55c724e42b30 msgr2=0x55c724e48860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.994+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 shutdown_connections 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.995+0000 7f7f33398740 1 -- 172.21.15.37:0/3849256405 wait complete. 2024-04-23T18:29:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:41.995+0000 7f7f33398740 20 client.16587 trim_cache size 0 max 16384 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 Processor -- start 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 -- start start 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46620 0x55c724e5d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 0x55c724e4d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281b04b0 con 0x55c724e46f40 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281add50 con 0x55c724e46620 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f33398740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281abbc0 con 0x55c724e51800 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 0x55c724e4d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 0x55c724e4d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59190/0 (socket says 172.21.15.37:59190) 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 -- 172.21.15.37:0/3912893565 learned_addr learned my addr 172.21.15.37:0/3912893565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 -- 172.21.15.37:0/3912893565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 msgr2=0x55c724e4e850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 -- 172.21.15.37:0/3912893565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46620 msgr2=0x55c724e5d2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46620 0x55c724e5d2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 -- 172.21.15.37:0/3912893565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c724e45e20 con 0x55c724e46f40 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.010+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 0x55c724e4d370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7f180b8f80 tx=0x7f7f180bc720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6440a8ce113a9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f13fff640 1 -- 172.21.15.37:0/3912893565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18048070 con 0x55c724e46f40 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f13fff640 1 -- 172.21.15.37:0/3912893565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f180bd220 con 0x55c724e46f40 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f13fff640 1 -- 172.21.15.37:0/3912893565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180baeb0 con 0x55c724e46f40 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 -- 172.21.15.37:0/3912893565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 msgr2=0x55c724e4d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 0x55c724e4d370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7f180b8f80 tx=0x7f7f180bc720 comp rx=0 tx=0).stop 2024-04-23T18:29:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 -- 172.21.15.37:0/3912893565 shutdown_connections 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46f40 0x55c724e4d370 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 --2- 172.21.15.37:0/3912893565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46620 0x55c724e5d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 -- 172.21.15.37:0/3912893565 >> 172.21.15.37:0/3912893565 conn(0x55c724e42b30 msgr2=0x55c724e48860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 -- 172.21.15.37:0/3912893565 shutdown_connections 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.011+0000 7f7f33398740 1 -- 172.21.15.37:0/3912893565 wait complete. 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 -- start start 2024-04-23T18:29:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 Processor -- start 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281ac210 con 0x55c724e46620 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281ad680 con 0x55c724e46f40 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281ac8e0 con 0x55c724e51800 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f33398740 10 client.0 _ll_register_callbacks cb 0x7f7f2000bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e4af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59194/0 (socket says 172.21.15.37:59194) 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 -- 172.21.15.37:0/1678978412 learned_addr learned my addr 172.21.15.37:0/1678978412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2ec19640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 -- 172.21.15.37:0/1678978412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 msgr2=0x55c724e4ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 -- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 msgr2=0x55c724e4b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 -- 172.21.15.37:0/1678978412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c724ec6b70 con 0x55c724e46620 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.012+0000 7f7f2e418640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e4af90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005fd0 tx=0x7f7f24012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ca19d5f7a4b455 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f11ffb640 1 -- 172.21.15.37:0/1678978412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2401f030 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f11ffb640 1 -- 172.21.15.37:0/1678978412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f7f2401b5b0 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f180bd220 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f11ffb640 1 -- 172.21.15.37:0/1678978412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f2401b7b0 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 10 client.16593 fetch_fsmap learned FSMap version 39 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c724ec7d80 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f11ffb640 1 -- 172.21.15.37:0/1678978412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f180bd220 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f11ffb640 1 -- 172.21.15.37:0/1678978412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8342+0+0 (secure 0 0 0) 0x7f7f2400f040 con 0x55c724e46620 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 10 client.16593 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 20 client.16593 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724ec8530 0x55c724eca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 4 client.16593 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.013+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c724db3200 con 0x55c724ec8530 2024-04-23T18:29:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.014+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724ec8530 0x55c724eca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.014+0000 7f7f2dc17640 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724ec8530 0x55c724eca950 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b328316b5d6437 server_cookie=77227e8a96bb35fb in_seq=0 out_seq=0 2024-04-23T18:29:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.014+0000 7f7f11ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f11ffb640 1 -- 172.21.15.37:0/1678978412 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7f180bde50 con 0x55c724ec8530 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f11ffb640 10 client.16593 handle_command_reply: tid=0 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 client.16593 shutdown 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 msgr2=0x55c724e4af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e4af90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005fd0 tx=0x7f7f24012820 comp rx=0 tx=0).stop 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 shutdown_connections 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c724ec8530 0x55c724eca950 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c724e51800 0x55c724e4ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c724e46f40 0x55c724e4b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f11ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 --2- 172.21.15.37:0/1678978412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c724e46620 0x55c724e4af90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 >> 172.21.15.37:0/1678978412 conn(0x55c724e42b30 msgr2=0x55c724e491b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 shutdown_connections 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 1 -- 172.21.15.37:0/1678978412 wait complete. 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.047+0000 7f7f33398740 20 client.16593 trim_cache size 0 max 16384 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:29:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:29:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f140a3e40 msgr2=0x7f7f140a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f140a3e40 0x7f7f140a6300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f2001d000 tx=0x7f7f20017000 comp rx=0 tx=0).stop 2024-04-23T18:29:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 msgr2=0x7f7f28198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 0x7f7f28198c10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7f18004770 tx=0x7f7f180042f0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 shutdown_connections 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f7f140a3e40 0x7f7f140a6300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28106fe0 0x7f7f2819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f28101b40 0x7f7f28198c10 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 --2- 172.21.15.37:0/1563797898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f281011a0 0x7f7f281986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 >> 172.21.15.37:0/1563797898 conn(0x7f7f28076b80 msgr2=0x7f7f280fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 shutdown_connections 2024-04-23T18:29:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:42.048+0000 7f7f3067c640 1 -- 172.21.15.37:0/1563797898 wait complete. 2024-04-23T18:29:42.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:42 smithi172 ceph-mon[32456]: pgmap v287: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:29:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:42 smithi173 ceph-mon[32906]: pgmap v287: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:29:42.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:42 smithi037 ceph-mon[29404]: pgmap v287: 97 pgs: 97 active+clean; 277 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-04-23T18:29:42.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 457698 3.39 MB/sec execute 347 sec latency 741.832 ms 2024-04-23T18:29:43.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 458344 3.39 MB/sec execute 348 sec latency 48.204 ms 2024-04-23T18:29:44.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:44 smithi172 ceph-mon[32456]: pgmap v288: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-04-23T18:29:44.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:44 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:44 smithi173 ceph-mon[32906]: pgmap v288: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-04-23T18:29:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:44 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:44 smithi037 ceph-mon[29404]: pgmap v288: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-04-23T18:29:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:44 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:44.840 INFO:tasks.workunit.client.0.smithi037.stdout: 1 459722 3.45 MB/sec execute 349 sec latency 54.971 ms 2024-04-23T18:29:45.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:45 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:45 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:45.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:45 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:29:45.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:45 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:45.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:45 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:45.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:45 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:29:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:45 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:45 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:45.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:45 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:29:45.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 460030 3.44 MB/sec execute 350 sec latency 804.577 ms 2024-04-23T18:29:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:46 smithi173 ceph-mon[32906]: pgmap v289: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-04-23T18:29:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:46 smithi037 ceph-mon[29404]: pgmap v289: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-04-23T18:29:46.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:46 smithi172 ceph-mon[32456]: pgmap v289: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-04-23T18:29:46.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 460164 3.43 MB/sec execute 351 sec latency 1752.861 ms 2024-04-23T18:29:47.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 462296 3.46 MB/sec execute 352 sec latency 24.563 ms 2024-04-23T18:29:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:48 smithi173 ceph-mon[32906]: pgmap v290: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-04-23T18:29:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:48 smithi037 ceph-mon[29404]: pgmap v290: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-04-23T18:29:48.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:48 smithi172 ceph-mon[32456]: pgmap v290: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-04-23T18:29:48.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 463595 3.47 MB/sec execute 353 sec latency 221.133 ms 2024-04-23T18:29:49.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 463595 3.46 MB/sec execute 354 sec latency 1221.268 ms 2024-04-23T18:29:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:50 smithi173 ceph-mon[32906]: pgmap v291: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-04-23T18:29:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:50 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:29:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:50 smithi037 ceph-mon[29404]: pgmap v291: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-04-23T18:29:50.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:50 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:29:50.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:50 smithi172 ceph-mon[32456]: pgmap v291: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-04-23T18:29:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:50 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:29:50.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 463595 3.45 MB/sec execute 355 sec latency 2221.377 ms 2024-04-23T18:29:51.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 463595 3.44 MB/sec execute 356 sec latency 3221.480 ms 2024-04-23T18:29:52.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:52 smithi037 ceph-mon[29404]: pgmap v292: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-04-23T18:29:52.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:52 smithi172 ceph-mon[32456]: pgmap v292: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-04-23T18:29:52.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:52 smithi173 ceph-mon[32906]: pgmap v292: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-04-23T18:29:52.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 464320 3.43 MB/sec execute 357 sec latency 3962.147 ms 2024-04-23T18:29:53.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 465994 3.46 MB/sec execute 358 sec latency 13.259 ms 2024-04-23T18:29:54.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:54 smithi037 ceph-mon[29404]: pgmap v293: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-04-23T18:29:54.405 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:54 smithi173 ceph-mon[32906]: pgmap v293: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-04-23T18:29:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:54 smithi172 ceph-mon[32456]: pgmap v293: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-04-23T18:29:54.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 467138 3.46 MB/sec execute 359 sec latency 288.045 ms 2024-04-23T18:29:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 -- 172.21.15.37:0/877148695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8090107290 msgr2=0x7f809010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 --2- 172.21.15.37:0/877148695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8090107290 0x7f809010ee60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8088009ca0 tx=0x7f808802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 -- 172.21.15.37:0/877148695 shutdown_connections 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 --2- 172.21.15.37:0/877148695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8090107290 0x7f809010ee60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 --2- 172.21.15.37:0/877148695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090102b70 0x7f8090106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 --2- 172.21.15.37:0/877148695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80901021c0 0x7f80901025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 -- 172.21.15.37:0/877148695 >> 172.21.15.37:0/877148695 conn(0x7f80900fda20 msgr2=0x7f80900ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 -- 172.21.15.37:0/877148695 shutdown_connections 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 -- 172.21.15.37:0/877148695 wait complete. 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 Processor -- start 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.147+0000 7f8096bf9640 1 -- start start 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901021c0 0x7f80901986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090107290 0x7f8090198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 0x7f809019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8090112200 con 0x7f809019cf10 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8090112080 con 0x7f80901021c0 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8090112380 con 0x7f8090107290 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 0x7f809019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 0x7f809019d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55436/0 (socket says 172.21.15.37:55436) 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 -- 172.21.15.37:0/937457575 learned_addr learned my addr 172.21.15.37:0/937457575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8094995640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901021c0 0x7f80901986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f808ffff640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090107290 0x7f8090198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 -- 172.21.15.37:0/937457575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090107290 msgr2=0x7f8090198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090107290 0x7f8090198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 -- 172.21.15.37:0/937457575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901021c0 msgr2=0x7f80901986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901021c0 0x7f80901986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 -- 172.21.15.37:0/937457575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809019da00 con 0x7f809019cf10 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8095196640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 0x7f809019d3c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8088005cc0 tx=0x7f808802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f808dffb640 1 -- 172.21.15.37:0/937457575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088003bf0 con 0x7f809019cf10 2024-04-23T18:29:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f808dffb640 1 -- 172.21.15.37:0/937457575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8088003d90 con 0x7f809019cf10 2024-04-23T18:29:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809019dc90 con 0x7f809019cf10 2024-04-23T18:29:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.148+0000 7f808dffb640 1 -- 172.21.15.37:0/937457575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808802fbc0 con 0x7f809019cf10 2024-04-23T18:29:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.149+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80901a54b0 con 0x7f809019cf10 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f808dffb640 1 -- 172.21.15.37:0/937457575 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f808802fd60 con 0x7f809019cf10 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 Processor -- start 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 -- start start 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e74105a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e7410696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e74106c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e740fbf970 con 0x55e74105a540 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e740f5c4a0 con 0x55e741069c10 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.151+0000 7f80999df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e740fbb930 con 0x55e74105ae60 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e74105a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808dffb640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f80780a3dc0 0x7f80780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e74105a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55446/0 (socket says 172.21.15.37:55446) 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 -- 172.21.15.37:0/3761471100 learned_addr learned my addr 172.21.15.37:0/3761471100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e74106c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f8095196640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f80780a3dc0 0x7f80780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f8094995640 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e7410696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 -- 172.21.15.37:0/3761471100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 msgr2=0x55e7410696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e7410696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 -- 172.21.15.37:0/3761471100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 msgr2=0x55e74106c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e74106c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 -- 172.21.15.37:0/3761471100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e740cd2210 con 0x55e74105a540 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808dffb640 1 -- 172.21.15.37:0/937457575 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f80880f1e00 con 0x7f809019cf10 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f808ffff640 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e74105a920 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f80840058a0 tx=0x7f808400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbacd43d70944419 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f8095196640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f80780a3dc0 0x7f80780a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8088090b10 tx=0x7f8088090b40 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f8076ffd640 1 -- 172.21.15.37:0/3761471100 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004030 con 0x55e74105a540 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f8076ffd640 1 -- 172.21.15.37:0/3761471100 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80840054d0 con 0x55e74105a540 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.152+0000 7f8076ffd640 1 -- 172.21.15.37:0/3761471100 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004a90 con 0x55e74105a540 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 -- 172.21.15.37:0/3761471100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 msgr2=0x55e74105a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e74105a920 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f80840058a0 tx=0x7f808400b780 comp rx=0 tx=0).stop 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 -- 172.21.15.37:0/3761471100 shutdown_connections 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e74106c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e7410696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 --2- 172.21.15.37:0/3761471100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e74105a920 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 -- 172.21.15.37:0/3761471100 >> 172.21.15.37:0/3761471100 conn(0x55e741056b30 msgr2=0x55e741056f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 -- 172.21.15.37:0/3761471100 shutdown_connections 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.153+0000 7f80999df740 1 -- 172.21.15.37:0/3761471100 wait complete. 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 -- start start 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 Processor -- start 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105a540 0x55e741069530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 0x55e741066820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741066d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e74106c840 con 0x55e74105ae60 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e740f5a0b0 con 0x55e741069c10 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e74106c9c0 con 0x55e74105a540 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f80999df740 10 client.0 _ll_register_callbacks cb 0x55e740f47d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f8094995640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 0x55e741066820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f8094995640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 0x55e741066820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55448/0 (socket says 172.21.15.37:55448) 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f8094995640 1 -- 172.21.15.37:0/61069095 learned_addr learned my addr 172.21.15.37:0/61069095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.154+0000 7f808ffff640 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741066d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f8094995640 1 -- 172.21.15.37:0/61069095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105a540 msgr2=0x55e741069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f8094995640 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105a540 0x55e741069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f8094995640 1 -- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 msgr2=0x55e741066d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f8094995640 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741066d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f8094995640 1 -- 172.21.15.37:0/61069095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e740d9e620 con 0x55e74105ae60 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f8094995640 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 0x55e741066820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f808000b960 tx=0x7f808000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51adaf5ffcf42c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f806ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f806ffff640 1 -- 172.21.15.37:0/61069095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080003ea0 con 0x55e74105ae60 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f806ffff640 1 -- 172.21.15.37:0/61069095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f8080005220 con 0x55e74105ae60 2024-04-23T18:29:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80840054d0 con 0x55e74105ae60 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f806ffff640 1 -- 172.21.15.37:0/61069095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8080005420 con 0x55e74105ae60 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f80999df740 10 client.16605 fetch_fsmap learned FSMap version 40 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e7410787c0 con 0x55e74105ae60 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.155+0000 7f806ffff640 1 -- 172.21.15.37:0/61069095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80840054d0 con 0x55e74105ae60 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f806ffff640 1 -- 172.21.15.37:0/61069095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f8080019030 con 0x55e74105ae60 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f80999df740 10 client.16605 fetch_fsmap finished waiting for FSMap version 40 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f80999df740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f80999df740 20 client.16605 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f80999df740 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e741079cd0 0x55e74107c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f80999df740 4 client.16605 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e740fcd390 con 0x55e741079cd0 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.156+0000 7f808ffff640 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e741079cd0 0x55e74107c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.157+0000 7f806ffff640 10 client.16605 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.157+0000 7f808ffff640 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e741079cd0 0x55e74107c120 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6f06f2fea82219 server_cookie=9a13d49baf89b2af in_seq=0 out_seq=0 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.157+0000 7f806ffff640 1 -- 172.21.15.37:0/61069095 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e740fcd390 con 0x55e741079cd0 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.157+0000 7f806ffff640 10 client.16605 handle_command_reply: tid=0 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.157+0000 7f80999df740 1 client.16605 shutdown 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 msgr2=0x55e741066820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 0x55e741066820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f808000b960 tx=0x7f808000be20 comp rx=0 tx=0).stop 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 shutdown_connections 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e741079cd0 0x55e74107c120 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741066d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f806ffff640 0 client.16605 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105ae60 0x55e741066820 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 --2- 172.21.15.37:0/61069095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105a540 0x55e741069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 >> 172.21.15.37:0/61069095 conn(0x55e741056b30 msgr2=0x55e74105c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 shutdown_connections 2024-04-23T18:29:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 1 -- 172.21.15.37:0/61069095 wait complete. 2024-04-23T18:29:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.158+0000 7f80999df740 20 client.16605 trim_cache size 0 max 16384 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 Processor -- start 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 -- start start 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e741075700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e741061d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80901ae310 con 0x55e74105a540 2024-04-23T18:29:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80901abb80 con 0x55e741069c10 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f80999df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80901a99f0 con 0x55e74105ae60 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e741075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e741075700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55450/0 (socket says 172.21.15.37:55450) 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 -- 172.21.15.37:0/3109618121 learned_addr learned my addr 172.21.15.37:0/3109618121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 -- 172.21.15.37:0/3109618121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 msgr2=0x55e741061d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e741061d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 -- 172.21.15.37:0/3109618121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 msgr2=0x55e741069060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741069060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 -- 172.21.15.37:0/3109618121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e741056960 con 0x55e74105a540 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.173+0000 7f808ffff640 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e741075700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f80840044d0 tx=0x7f80840045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5972647d647ea720 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80767fc640 1 -- 172.21.15.37:0/3109618121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808400bca0 con 0x55e74105a540 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80767fc640 1 -- 172.21.15.37:0/3109618121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80840298e0 con 0x55e74105a540 2024-04-23T18:29:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80767fc640 1 -- 172.21.15.37:0/3109618121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004740 con 0x55e74105a540 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 -- 172.21.15.37:0/3109618121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 msgr2=0x55e741075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e741075700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f80840044d0 tx=0x7f80840045a0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 -- 172.21.15.37:0/3109618121 shutdown_connections 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e741069c10 0x55e741069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e741061d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 --2- 172.21.15.37:0/3109618121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e74105a540 0x55e741075700 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 -- 172.21.15.37:0/3109618121 >> 172.21.15.37:0/3109618121 conn(0x55e741056b30 msgr2=0x55e741061670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 -- 172.21.15.37:0/3109618121 shutdown_connections 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.174+0000 7f80999df740 1 -- 172.21.15.37:0/3109618121 wait complete. 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 -- start start 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 Processor -- start 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e74105a540 0x55e741071e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e74106d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 0x55e74106d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80901aa040 con 0x55e741069c10 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80901ab4b0 con 0x55e74105a540 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80901aa710 con 0x55e74105ae60 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f80999df740 10 client.0 _ll_register_callbacks cb 0x7f8080004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f808ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 0x55e74106d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f808ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 0x55e74106d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55464/0 (socket says 172.21.15.37:55464) 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.175+0000 7f808ffff640 1 -- 172.21.15.37:0/283540150 learned_addr learned my addr 172.21.15.37:0/283540150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f808ffff640 1 -- 172.21.15.37:0/283540150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 msgr2=0x55e74106d1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f808ffff640 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e74106d1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f808ffff640 1 -- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e74105a540 msgr2=0x55e741071e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f808ffff640 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e74105a540 0x55e741071e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f808ffff640 1 -- 172.21.15.37:0/283540150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74106dd00 con 0x55e741069c10 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f808ffff640 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 0x55e74106d710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f808400bb70 tx=0x7f8084004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcce9b24cd1f968e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f806ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f806ffff640 1 -- 172.21.15.37:0/283540150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084005590 con 0x55e741069c10 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f806ffff640 1 -- 172.21.15.37:0/283540150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f8084029430 con 0x55e741069c10 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80840298e0 con 0x55e741069c10 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f806ffff640 1 -- 172.21.15.37:0/283540150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8084029630 con 0x55e741069c10 2024-04-23T18:29:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f80999df740 10 client.16611 fetch_fsmap learned FSMap version 40 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e7410e00d0 con 0x55e741069c10 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.176+0000 7f806ffff640 1 -- 172.21.15.37:0/283540150 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80840298e0 con 0x55e741069c10 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f806ffff640 1 -- 172.21.15.37:0/283540150 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 40) ==== 8346+0+0 (secure 0 0 0) 0x7f808401c020 con 0x55e741069c10 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f80999df740 10 client.16611 fetch_fsmap finished waiting for FSMap version 40 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f80999df740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f80999df740 20 client.16611 populate_metadata read hostname 'smithi037' 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f80999df740 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e7410e0670 0x55e7410e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f80999df740 4 client.16611 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.177+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e740fc7200 con 0x55e7410e0670 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.178+0000 7f8094995640 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e7410e0670 0x55e7410e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.179+0000 7f8094995640 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e7410e0670 0x55e7410e2a90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4fa381d9b48f3 server_cookie=701add12a6e61a15 in_seq=0 out_seq=0 2024-04-23T18:29:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.179+0000 7f806ffff640 10 client.16611 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f806ffff640 1 -- 172.21.15.37:0/283540150 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e740fc7200 con 0x55e7410e0670 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f806ffff640 10 client.16611 handle_command_reply: tid=0 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 client.16611 shutdown 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 msgr2=0x55e74106d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 0x55e74106d710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f808400bb70 tx=0x7f8084004740 comp rx=0 tx=0).stop 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 shutdown_connections 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55e7410e0670 0x55e7410e2a90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e741069c10 0x55e74106d710 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e74105ae60 0x55e74106d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 --2- 172.21.15.37:0/283540150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e74105a540 0x55e741071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f806ffff640 0 client.16611 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 >> 172.21.15.37:0/283540150 conn(0x55e741056b30 msgr2=0x55e741055420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 shutdown_connections 2024-04-23T18:29:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 1 -- 172.21.15.37:0/283540150 wait complete. 2024-04-23T18:29:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.215+0000 7f80999df740 20 client.16611 trim_cache size 0 max 16384 2024-04-23T18:29:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:29:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:29:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:29:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f80780a3dc0 msgr2=0x7f80780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f80780a3dc0 0x7f80780a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8088090b10 tx=0x7f8088090b40 comp rx=0 tx=0).stop 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 msgr2=0x7f809019d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 0x7f809019d3c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8088005cc0 tx=0x7f808802f380 comp rx=0 tx=0).stop 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 shutdown_connections 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f80780a3dc0 0x7f80780a6280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f809019cf10 0x7f809019d3c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090107290 0x7f8090198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 --2- 172.21.15.37:0/937457575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901021c0 0x7f80901986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 >> 172.21.15.37:0/937457575 conn(0x7f80900fda20 msgr2=0x7f80900fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 shutdown_connections 2024-04-23T18:29:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:29:55.216+0000 7f8096bf9640 1 -- 172.21.15.37:0/937457575 wait complete. 2024-04-23T18:29:55.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 468815 3.48 MB/sec execute 360 sec latency 648.613 ms 2024-04-23T18:29:56.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:56 smithi037 ceph-mon[29404]: pgmap v294: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-04-23T18:29:56.388 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:56 smithi172 ceph-mon[32456]: pgmap v294: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-04-23T18:29:56.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:56 smithi173 ceph-mon[32906]: pgmap v294: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-04-23T18:29:56.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 469819 3.48 MB/sec execute 361 sec latency 25.305 ms 2024-04-23T18:29:57.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 470685 3.49 MB/sec execute 362 sec latency 477.875 ms 2024-04-23T18:29:58.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:29:58 smithi172 ceph-mon[32456]: pgmap v295: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-04-23T18:29:58.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:29:58 smithi173 ceph-mon[32906]: pgmap v295: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-04-23T18:29:58.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:29:58 smithi037 ceph-mon[29404]: pgmap v295: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-04-23T18:29:58.841 INFO:tasks.workunit.client.0.smithi037.stdout: 1 471426 3.48 MB/sec execute 363 sec latency 1191.627 ms 2024-04-23T18:29:59.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 472974 3.51 MB/sec execute 364 sec latency 17.147 ms 2024-04-23T18:30:00.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:00 smithi172 ceph-mon[32456]: pgmap v296: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-23T18:30:00.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:00 smithi172 ceph-mon[32456]: overall HEALTH_OK 2024-04-23T18:30:00.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:00 smithi173 ceph-mon[32906]: pgmap v296: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-23T18:30:00.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:00 smithi173 ceph-mon[32906]: overall HEALTH_OK 2024-04-23T18:30:00.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:00 smithi037 ceph-mon[29404]: pgmap v296: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-23T18:30:00.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:00 smithi037 ceph-mon[29404]: overall HEALTH_OK 2024-04-23T18:30:00.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 474231 3.51 MB/sec execute 365 sec latency 142.709 ms 2024-04-23T18:30:01.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 474231 3.50 MB/sec execute 366 sec latency 1142.832 ms 2024-04-23T18:30:02.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:02 smithi172 ceph-mon[32456]: pgmap v297: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-23T18:30:02.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:02 smithi173 ceph-mon[32906]: pgmap v297: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-23T18:30:02.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:02 smithi037 ceph-mon[29404]: pgmap v297: 97 pgs: 97 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-23T18:30:02.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 474804 3.49 MB/sec execute 367 sec latency 1874.755 ms 2024-04-23T18:30:03.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 476370 3.52 MB/sec execute 368 sec latency 23.734 ms 2024-04-23T18:30:04.421 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:04 smithi037 ceph-mon[29404]: pgmap v298: 97 pgs: 97 active+clean; 294 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-04-23T18:30:04.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:04 smithi172 ceph-mon[32456]: pgmap v298: 97 pgs: 97 active+clean; 294 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-04-23T18:30:04.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:04 smithi173 ceph-mon[32906]: pgmap v298: 97 pgs: 97 active+clean; 294 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-04-23T18:30:04.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477774 3.52 MB/sec execute 369 sec latency 34.488 ms 2024-04-23T18:30:05.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:05 smithi172 ceph-mon[32456]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:30:05.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:05 smithi173 ceph-mon[32906]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:30:05.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:05 smithi037 ceph-mon[29404]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:30:05.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477780 3.51 MB/sec execute 370 sec latency 999.840 ms 2024-04-23T18:30:06.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:06 smithi172 ceph-mon[32456]: pgmap v300: 97 pgs: 97 active+clean; 294 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-04-23T18:30:06.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:06 smithi173 ceph-mon[32906]: pgmap v300: 97 pgs: 97 active+clean; 294 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-04-23T18:30:06.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:06 smithi037 ceph-mon[29404]: pgmap v300: 97 pgs: 97 active+clean; 294 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 166 op/s 2024-04-23T18:30:06.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477780 3.51 MB/sec execute 371 sec latency 1999.945 ms 2024-04-23T18:30:07.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477780 3.50 MB/sec execute 372 sec latency 3000.020 ms 2024-04-23T18:30:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 -- 172.21.15.37:0/3445224788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 msgr2=0x7f9070101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 --2- 172.21.15.37:0/3445224788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f9070101140 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9068009900 tx=0x7f906802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 -- 172.21.15.37:0/3445224788 shutdown_connections 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 --2- 172.21.15.37:0/3445224788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907010e910 0x7f9070110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 --2- 172.21.15.37:0/3445224788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 0x7f907010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 --2- 172.21.15.37:0/3445224788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f9070101140 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 -- 172.21.15.37:0/3445224788 >> 172.21.15.37:0/3445224788 conn(0x7f9070077fd0 msgr2=0x7f90700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 -- 172.21.15.37:0/3445224788 shutdown_connections 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 -- 172.21.15.37:0/3445224788 wait complete. 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.314+0000 7f9077d08640 1 Processor -- start 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 -- start start 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f90701a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 0x7f90701a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907010e910 0x7f907019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9070114430 con 0x7f9070100d60 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90701142b0 con 0x7f9070101680 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9077d08640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90701145b0 con 0x7f907010e910 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f90701a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f90701a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41634/0 (socket says 172.21.15.37:41634) 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 -- 172.21.15.37:0/570091851 learned_addr learned my addr 172.21.15.37:0/570091851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f90762a5640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907010e910 0x7f907019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f90752a3640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 0x7f90701a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 -- 172.21.15.37:0/570091851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907010e910 msgr2=0x7f907019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907010e910 0x7f907019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 -- 172.21.15.37:0/570091851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 msgr2=0x7f90701a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 0x7f90701a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 -- 172.21.15.37:0/570091851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019af90 con 0x7f9070100d60 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f90752a3640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 0x7f90701a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.315+0000 7f9075aa4640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f90701a0610 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9068005ab0 tx=0x7f9068003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.316+0000 7f9066ffd640 1 -- 172.21.15.37:0/570091851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906803f070 con 0x7f9070100d60 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.316+0000 7f9066ffd640 1 -- 172.21.15.37:0/570091851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9068031d60 con 0x7f9070100d60 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.316+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019b220 con 0x7f9070100d60 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.316+0000 7f9066ffd640 1 -- 172.21.15.37:0/570091851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068035030 con 0x7f9070100d60 2024-04-23T18:30:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.316+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a7590 con 0x7f9070100d60 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.318+0000 7f9066ffd640 1 -- 172.21.15.37:0/570091851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9068042070 con 0x7f9070100d60 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.318+0000 7f907aa27740 1 Processor -- start 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 -- start start 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f88da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f8982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d9f7f2970 con 0x564d9f88d620 2024-04-23T18:30:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d9f78f4a0 con 0x564d9f898800 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f907aa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564d9f7ee930 con 0x564d9f88df40 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f9066ffd640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f905c0a3e40 0x7f905c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f90762a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f88da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f90762a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f88da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41640/0 (socket says 172.21.15.37:41640) 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f90762a5640 1 -- 172.21.15.37:0/2306929180 learned_addr learned my addr 172.21.15.37:0/2306929180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f90762a5640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f905c0a3e40 0x7f905c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f90752a3640 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f9066ffd640 1 -- 172.21.15.37:0/570091851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8623+0+0 (secure 0 0 0) 0x7f90680f1130 con 0x7f9070100d60 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.319+0000 7f9075aa4640 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f8982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 -- 172.21.15.37:0/2306929180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 msgr2=0x564d9f8982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f8982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 -- 172.21.15.37:0/2306929180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 msgr2=0x564d9f89abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 -- 172.21.15.37:0/2306929180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9f505210 con 0x564d9f88d620 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90752a3640 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f88da00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f906c012ce0 tx=0x7f906c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bad7f2ce842e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f90762a5640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f905c0a3e40 0x7f905c0a6300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c00c000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f905bfff640 1 -- 172.21.15.37:0/2306929180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c023070 con 0x564d9f88d620 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f905bfff640 1 -- 172.21.15.37:0/2306929180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c0124f0 con 0x564d9f88d620 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.320+0000 7f905bfff640 1 -- 172.21.15.37:0/2306929180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00ce20 con 0x564d9f88d620 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 -- 172.21.15.37:0/2306929180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 msgr2=0x564d9f88da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f88da00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f906c012ce0 tx=0x7f906c010740 comp rx=0 tx=0).stop 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 -- 172.21.15.37:0/2306929180 shutdown_connections 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f8982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 --2- 172.21.15.37:0/2306929180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f88da00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 -- 172.21.15.37:0/2306929180 >> 172.21.15.37:0/2306929180 conn(0x564d9f889b30 msgr2=0x564d9f889f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 -- 172.21.15.37:0/2306929180 shutdown_connections 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 -- 172.21.15.37:0/2306929180 wait complete. 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 -- start start 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.321+0000 7f907aa27740 1 Processor -- start 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f895490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 0x564d9f895a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d9f89b3d0 con 0x564d9f898800 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d9f78d0b0 con 0x564d9f88d620 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564d9f89b550 con 0x564d9f88df40 2024-04-23T18:30:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f907aa27740 10 client.0 _ll_register_callbacks cb 0x564d9f77ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 0x564d9f895a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 0x564d9f895a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41646/0 (socket says 172.21.15.37:41646) 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 -- 172.21.15.37:0/98325872 learned_addr learned my addr 172.21.15.37:0/98325872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90752a3640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f895490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f9075aa4640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 -- 172.21.15.37:0/98325872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 msgr2=0x564d9f895490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f895490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 -- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 msgr2=0x564d9f8981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.322+0000 7f90762a5640 1 -- 172.21.15.37:0/98325872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9f5d1620 con 0x564d9f898800 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90762a5640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 0x564d9f895a30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f906c012cb0 tx=0x7f906c012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580e036b1fce607f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90597fa640 1 -- 172.21.15.37:0/98325872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c023040 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90597fa640 1 -- 172.21.15.37:0/98325872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8623+0+0 (secure 0 0 0) 0x7f906c00cdd0 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c0124f0 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90597fa640 1 -- 172.21.15.37:0/98325872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f906c004a00 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f907aa27740 10 client.16623 fetch_fsmap learned FSMap version 41 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564d9f8a73b0 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90597fa640 1 -- 172.21.15.37:0/98325872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c0124f0 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f90597fa640 1 -- 172.21.15.37:0/98325872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f906c00f040 con 0x564d9f898800 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f907aa27740 10 client.16623 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.323+0000 7f907aa27740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f907aa27740 20 client.16623 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f907aa27740 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f8a8870 0x564d9f8aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f907aa27740 4 client.16623 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d9f800390 con 0x564d9f8a8870 2024-04-23T18:30:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f9075aa4640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f8a8870 0x564d9f8aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f9075aa4640 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f8a8870 0x564d9f8aac60 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37882341376571b2 server_cookie=520786650ff3e2ef in_seq=0 out_seq=0 2024-04-23T18:30:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.324+0000 7f90597fa640 10 client.16623 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f90597fa640 1 -- 172.21.15.37:0/98325872 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d9f800390 con 0x564d9f8a8870 2024-04-23T18:30:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f90597fa640 10 client.16623 handle_command_reply: tid=0 2024-04-23T18:30:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 client.16623 shutdown 2024-04-23T18:30:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 msgr2=0x564d9f895a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 0x564d9f895a30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f906c012cb0 tx=0x7f906c012300 comp rx=0 tx=0).stop 2024-04-23T18:30:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 shutdown_connections 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f8a8870 0x564d9f8aac60 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f898800 0x564d9f895a30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f90597fa640 0 client.16623 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f895490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 --2- 172.21.15.37:0/98325872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 >> 172.21.15.37:0/98325872 conn(0x564d9f889b30 msgr2=0x564d9f88f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 shutdown_connections 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.325+0000 7f907aa27740 1 -- 172.21.15.37:0/98325872 wait complete. 2024-04-23T18:30:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.326+0000 7f907aa27740 20 client.16623 trim_cache size 0 max 16384 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.340+0000 7f907aa27740 1 Processor -- start 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 -- start start 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 0x564d9f897cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f898800 0x564d9f8b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b27d0 con 0x564d9f88df40 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b0040 con 0x564d9f88d620 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f907aa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90701adeb0 con 0x564d9f898800 2024-04-23T18:30:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 0x564d9f897cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f9075aa4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 0x564d9f897cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41650/0 (socket says 172.21.15.37:41650) 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 -- 172.21.15.37:0/3514602616 learned_addr learned my addr 172.21.15.37:0/3514602616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90762a5640 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f898800 0x564d9f8b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 -- 172.21.15.37:0/3514602616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f898800 msgr2=0x564d9f8b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f898800 0x564d9f8b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 -- 172.21.15.37:0/3514602616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 msgr2=0x564d9f8a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 -- 172.21.15.37:0/3514602616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9f8939e0 con 0x564d9f88df40 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f9075aa4640 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.341+0000 7f90752a3640 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 0x564d9f897cd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f907019bf20 tx=0x7f9060011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b9e8cf5de18f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f905b7fe640 1 -- 172.21.15.37:0/3514602616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060004030 con 0x564d9f88df40 2024-04-23T18:30:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f905b7fe640 1 -- 172.21.15.37:0/3514602616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9060005020 con 0x564d9f88df40 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f905b7fe640 1 -- 172.21.15.37:0/3514602616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060004ce0 con 0x564d9f88df40 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 -- 172.21.15.37:0/3514602616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 msgr2=0x564d9f897cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 0x564d9f897cd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f907019bf20 tx=0x7f9060011810 comp rx=0 tx=0).stop 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 -- 172.21.15.37:0/3514602616 shutdown_connections 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f898800 0x564d9f8b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88df40 0x564d9f897cd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 --2- 172.21.15.37:0/3514602616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f88d620 0x564d9f8a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 -- 172.21.15.37:0/3514602616 >> 172.21.15.37:0/3514602616 conn(0x564d9f889b30 msgr2=0x564d9f888cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 -- 172.21.15.37:0/3514602616 shutdown_connections 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.342+0000 7f907aa27740 1 -- 172.21.15.37:0/3514602616 wait complete. 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 -- start start 2024-04-23T18:30:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 Processor -- start 2024-04-23T18:30:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f8a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f89c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ae500 con 0x564d9f88d620 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90701af970 con 0x564d9f898800 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90701aebd0 con 0x564d9f88df40 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f907aa27740 10 client.0 _ll_register_callbacks cb 0x7f906c012ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f9075aa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f8a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f9075aa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f8a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41652/0 (socket says 172.21.15.37:41652) 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f9075aa4640 1 -- 172.21.15.37:0/1003870015 learned_addr learned my addr 172.21.15.37:0/1003870015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.343+0000 7f90752a3640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f89c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90762a5640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f9075aa4640 1 -- 172.21.15.37:0/1003870015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 msgr2=0x564d9f89c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f9075aa4640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f89c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f9075aa4640 1 -- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 msgr2=0x564d9f89c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f9075aa4640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f9075aa4640 1 -- 172.21.15.37:0/1003870015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9f90dc10 con 0x564d9f88d620 2024-04-23T18:30:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90762a5640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f9075aa4640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f8a0d10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f90680ee820 tx=0x7f906802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf2e884a1371a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90597fa640 1 -- 172.21.15.37:0/1003870015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068049040 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90597fa640 1 -- 172.21.15.37:0/1003870015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8623+0+0 (secure 0 0 0) 0x7f90680b9680 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9060005020 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90597fa640 1 -- 172.21.15.37:0/1003870015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f90680c5c80 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 10 client.16629 fetch_fsmap learned FSMap version 41 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564d9f90ee20 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90597fa640 1 -- 172.21.15.37:0/1003870015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9060005020 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f90597fa640 1 -- 172.21.15.37:0/1003870015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f90680bd270 con 0x564d9f88d620 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 10 client.16629 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 20 client.16629 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.344+0000 7f907aa27740 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f90f520 0x564d9f911940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.345+0000 7f907aa27740 4 client.16629 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.345+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d9f7fa200 con 0x564d9f90f520 2024-04-23T18:30:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.345+0000 7f90752a3640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f90f520 0x564d9f911940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.345+0000 7f90597fa640 10 client.16629 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.345+0000 7f90752a3640 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f90f520 0x564d9f911940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b14acf7c87f003f server_cookie=84a7ba382ccb1eb in_seq=0 out_seq=0 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f90597fa640 1 -- 172.21.15.37:0/1003870015 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d9f7fa200 con 0x564d9f90f520 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f90597fa640 10 client.16629 handle_command_reply: tid=0 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 client.16629 shutdown 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 msgr2=0x564d9f8a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f8a0d10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f90680ee820 tx=0x7f906802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 shutdown_connections 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564d9f90f520 0x564d9f911940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d9f898800 0x564d9f89c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d9f88df40 0x564d9f89c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f90597fa640 0 client.16629 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 --2- 172.21.15.37:0/1003870015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d9f88d620 0x564d9f8a0d10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 >> 172.21.15.37:0/1003870015 conn(0x564d9f889b30 msgr2=0x564d9f8a32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 shutdown_connections 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 1 -- 172.21.15.37:0/1003870015 wait complete. 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.381+0000 7f907aa27740 20 client.16629 trim_cache size 0 max 16384 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:30:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:30:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f905c0a3e40 msgr2=0x7f905c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f905c0a3e40 0x7f905c0a6300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c00c000 comp rx=0 tx=0).stop 2024-04-23T18:30:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 msgr2=0x7f90701a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f90701a0610 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9068005ab0 tx=0x7f9068003cd0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 shutdown_connections 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f905c0a3e40 0x7f905c0a6300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907010e910 0x7f907019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9070101680 0x7f90701a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 --2- 172.21.15.37:0/570091851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9070100d60 0x7f90701a0610 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 >> 172.21.15.37:0/570091851 conn(0x7f9070077fd0 msgr2=0x7f907010fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 shutdown_connections 2024-04-23T18:30:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:08.382+0000 7f9077d08640 1 -- 172.21.15.37:0/570091851 wait complete. 2024-04-23T18:30:08.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:08 smithi172 ceph-mon[32456]: pgmap v301: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:08.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:08 smithi173 ceph-mon[32906]: pgmap v301: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:08.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:08 smithi037 ceph-mon[29404]: pgmap v301: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:08.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.49 MB/sec execute 373 sec latency 3525.288 ms 2024-04-23T18:30:09.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:09 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:30:09.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:09 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:30:09.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:09 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:30:09.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.48 MB/sec execute 374 sec latency 1447.009 ms 2024-04-23T18:30:10.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:10 smithi172 ceph-mon[32456]: pgmap v302: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:10.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:10 smithi173 ceph-mon[32906]: pgmap v302: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:10.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:10 smithi037 ceph-mon[29404]: pgmap v302: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:10.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.47 MB/sec execute 375 sec latency 2447.133 ms 2024-04-23T18:30:11.842 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.46 MB/sec execute 376 sec latency 3447.248 ms 2024-04-23T18:30:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:11 smithi172 ceph-mon[32456]: pgmap v303: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:11 smithi172 ceph-mon[32456]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:30:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:11 smithi173 ceph-mon[32906]: pgmap v303: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:11 smithi173 ceph-mon[32906]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:30:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:11 smithi037 ceph-mon[29404]: pgmap v303: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-23T18:30:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:11 smithi037 ceph-mon[29404]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:30:12.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.45 MB/sec execute 377 sec latency 4447.365 ms 2024-04-23T18:30:13.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.44 MB/sec execute 378 sec latency 5447.484 ms 2024-04-23T18:30:14.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:13 smithi172 ceph-mon[32456]: pgmap v305: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-23T18:30:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:13 smithi173 ceph-mon[32906]: pgmap v305: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-23T18:30:14.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:13 smithi037 ceph-mon[29404]: pgmap v305: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-04-23T18:30:14.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.43 MB/sec execute 379 sec latency 6447.611 ms 2024-04-23T18:30:15.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.42 MB/sec execute 380 sec latency 7447.672 ms 2024-04-23T18:30:16.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:15 smithi172 ceph-mon[32456]: pgmap v306: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-04-23T18:30:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:15 smithi173 ceph-mon[32906]: pgmap v306: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-04-23T18:30:16.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:15 smithi037 ceph-mon[29404]: pgmap v306: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-04-23T18:30:16.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 477794 3.41 MB/sec execute 381 sec latency 8447.755 ms 2024-04-23T18:30:17.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 478563 3.41 MB/sec execute 382 sec latency 9187.108 ms 2024-04-23T18:30:18.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:17 smithi172 ceph-mon[32456]: pgmap v307: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:17 smithi173 ceph-mon[32906]: pgmap v307: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:18.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:17 smithi037 ceph-mon[29404]: pgmap v307: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:18.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 479898 3.43 MB/sec execute 383 sec latency 380.178 ms 2024-04-23T18:30:19.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 481198 3.44 MB/sec execute 384 sec latency 20.438 ms 2024-04-23T18:30:20.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:19 smithi172 ceph-mon[32456]: pgmap v308: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:19 smithi173 ceph-mon[32906]: pgmap v308: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:20.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:19 smithi037 ceph-mon[29404]: pgmap v308: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:20.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 481360 3.43 MB/sec execute 385 sec latency 919.603 ms 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 -- 172.21.15.37:0/2578435969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103b20 msgr2=0x7fb5d410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 --2- 172.21.15.37:0/2578435969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103b20 0x7fb5d410c000 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009900 tx=0x7fb5c802a480 comp rx=0 tx=0).stop 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 -- 172.21.15.37:0/2578435969 shutdown_connections 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 --2- 172.21.15.37:0/2578435969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d410c5d0 0x7fb5d410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 --2- 172.21.15.37:0/2578435969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103b20 0x7fb5d410c000 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 --2- 172.21.15.37:0/2578435969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d4103200 0x7fb5d41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 -- 172.21.15.37:0/2578435969 >> 172.21.15.37:0/2578435969 conn(0x7fb5d40771d0 msgr2=0x7fb5d40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 -- 172.21.15.37:0/2578435969 shutdown_connections 2024-04-23T18:30:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 -- 172.21.15.37:0/2578435969 wait complete. 2024-04-23T18:30:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.480+0000 7fb5daa23640 1 Processor -- start 2024-04-23T18:30:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 -- start start 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 0x7fb5d4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d4103b20 0x7fb5d4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d410c5d0 0x7fb5d419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4112210 con 0x7fb5d4103200 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4112090 con 0x7fb5d4103b20 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5daa23640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4112390 con 0x7fb5d410c5d0 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 0x7fb5d4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 0x7fb5d4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45580/0 (socket says 172.21.15.37:45580) 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d3fff640 1 -- 172.21.15.37:0/166947252 learned_addr learned my addr 172.21.15.37:0/166947252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d410c5d0 0x7fb5d419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d4103b20 0x7fb5d4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d3fff640 1 -- 172.21.15.37:0/166947252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d410c5d0 msgr2=0x7fb5d419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.481+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d410c5d0 0x7fb5d419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d3fff640 1 -- 172.21.15.37:0/166947252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d4103b20 msgr2=0x7fb5d4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d4103b20 0x7fb5d4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d3fff640 1 -- 172.21.15.37:0/166947252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d419d700 con 0x7fb5d4103200 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 0x7fb5d4198710 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400d770 tx=0x7fb5c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d17fa640 1 -- 172.21.15.37:0/166947252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4017070 con 0x7fb5d4103200 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d17fa640 1 -- 172.21.15.37:0/166947252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c40054d0 con 0x7fb5d4103200 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5d17fa640 1 -- 172.21.15.37:0/166947252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4004c30 con 0x7fb5d4103200 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d419d9f0 con 0x7fb5d4103200 2024-04-23T18:30:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.482+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41a5280 con 0x7fb5d4103200 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5d17fa640 1 -- 172.21.15.37:0/166947252 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5c4004e10 con 0x7fb5d4103200 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 Processor -- start 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 -- start start 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c83a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557847be8970 con 0x557847c8e800 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557847b854a0 con 0x557847c83f40 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.485+0000 7fb5dd815740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557847be4930 con 0x557847c83620 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d17fa640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb5bc0a3dc0 0x7fb5bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c90bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45582/0 (socket says 172.21.15.37:45582) 2024-04-23T18:30:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 -- 172.21.15.37:0/3961793756 learned_addr learned my addr 172.21.15.37:0/3961793756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c83a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb5bc0a3dc0 0x7fb5bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 -- 172.21.15.37:0/3961793756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 msgr2=0x557847c83a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c83a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d17fa640 1 -- 172.21.15.37:0/166947252 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fb5c40d0260 con 0x7fb5d4103200 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 -- 172.21.15.37:0/3961793756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 msgr2=0x557847c8e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 -- 172.21.15.37:0/3961793756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578478fb210 con 0x557847c8e800 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c83a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c90bf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009900 tx=0x7fb5c8032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41076e1f99bcfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb5bc0a3dc0 0x7fb5bc0a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009370 tx=0x7fb5cc0096a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5ba7fc640 1 -- 172.21.15.37:0/3961793756 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8040070 con 0x557847c8e800 2024-04-23T18:30:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5ba7fc640 1 -- 172.21.15.37:0/3961793756 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c8004030 con 0x557847c8e800 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.486+0000 7fb5ba7fc640 1 -- 172.21.15.37:0/3961793756 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8035040 con 0x557847c8e800 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 -- 172.21.15.37:0/3961793756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 msgr2=0x557847c90bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c90bf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009900 tx=0x7fb5c8032bd0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 -- 172.21.15.37:0/3961793756 shutdown_connections 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c90bf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 --2- 172.21.15.37:0/3961793756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c83a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 -- 172.21.15.37:0/3961793756 >> 172.21.15.37:0/3961793756 conn(0x557847c7fb30 msgr2=0x557847c7ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 -- 172.21.15.37:0/3961793756 shutdown_connections 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.487+0000 7fb5dd815740 1 -- 172.21.15.37:0/3961793756 wait complete. 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 -- start start 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 Processor -- start 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c8b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557847c913d0 con 0x557847c8e800 2024-04-23T18:30:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557847b830b0 con 0x557847c83f40 2024-04-23T18:30:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557847c91550 con 0x557847c83620 2024-04-23T18:30:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5dd815740 10 client.0 _ll_register_callbacks cb 0x557847b70d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5d8fc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c8b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5d8fc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c8b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45586/0 (socket says 172.21.15.37:45586) 2024-04-23T18:30:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5d8fc0640 1 -- 172.21.15.37:0/2445062662 learned_addr learned my addr 172.21.15.37:0/2445062662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.488+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d8fc0640 1 -- 172.21.15.37:0/2445062662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 msgr2=0x557847c8e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c8e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d8fc0640 1 -- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 msgr2=0x557847c8b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d8fc0640 1 -- 172.21.15.37:0/2445062662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578479c7620 con 0x557847c8e800 2024-04-23T18:30:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c8b960 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00f7a0 tx=0x7fb5cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72600da436e85a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5b37fe640 1 -- 172.21.15.37:0/2445062662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004d20 con 0x557847c8e800 2024-04-23T18:30:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5b37fe640 1 -- 172.21.15.37:0/2445062662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fb5cc01c8e0 con 0x557847c8e800 2024-04-23T18:30:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c8004030 con 0x557847c8e800 2024-04-23T18:30:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5b37fe640 1 -- 172.21.15.37:0/2445062662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb5cc01cae0 con 0x557847c8e800 2024-04-23T18:30:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5dd815740 10 client.16641 fetch_fsmap learned FSMap version 41 2024-04-23T18:30:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557847c9d2b0 con 0x557847c8e800 2024-04-23T18:30:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.489+0000 7fb5b37fe640 1 -- 172.21.15.37:0/2445062662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c8004030 con 0x557847c8e800 2024-04-23T18:30:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5b37fe640 1 -- 172.21.15.37:0/2445062662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7fb5cc02c030 con 0x557847c8e800 2024-04-23T18:30:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5dd815740 10 client.16641 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:30:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5dd815740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5dd815740 20 client.16641 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5dd815740 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847c9e830 0x557847ca0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5dd815740 4 client.16641 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557847bf6390 con 0x557847c9e830 2024-04-23T18:30:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847c9e830 0x557847ca0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847c9e830 0x557847ca0c40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2f2d4715d1cb67 server_cookie=3ab4c06c76b7e461 in_seq=0 out_seq=0 2024-04-23T18:30:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.490+0000 7fb5b37fe640 10 client.16641 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.491+0000 7fb5b37fe640 1 -- 172.21.15.37:0/2445062662 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557847bf6390 con 0x557847c9e830 2024-04-23T18:30:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.491+0000 7fb5b37fe640 10 client.16641 handle_command_reply: tid=0 2024-04-23T18:30:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.491+0000 7fb5dd815740 1 client.16641 shutdown 2024-04-23T18:30:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.491+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 msgr2=0x557847c8b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.491+0000 7fb5dd815740 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c8b960 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00f7a0 tx=0x7fb5cc004a40 comp rx=0 tx=0).stop 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 shutdown_connections 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847c9e830 0x557847ca0c40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c8e800 0x557847c8b960 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c8b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5b37fe640 0 client.16641 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 --2- 172.21.15.37:0/2445062662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83620 0x557847c8e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 >> 172.21.15.37:0/2445062662 conn(0x557847c7fb30 msgr2=0x557847c85860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 shutdown_connections 2024-04-23T18:30:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 1 -- 172.21.15.37:0/2445062662 wait complete. 2024-04-23T18:30:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.492+0000 7fb5dd815740 20 client.16641 trim_cache size 0 max 16384 2024-04-23T18:30:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 Processor -- start 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 -- start start 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83f40 0x557847c7eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c8e800 0x557847ca6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41b0580 con 0x557847c83620 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41addf0 con 0x557847c8e800 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5dd815740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41abc60 con 0x557847c83f40 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c9a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45598/0 (socket says 172.21.15.37:45598) 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 -- 172.21.15.37:0/1995186516 learned_addr learned my addr 172.21.15.37:0/1995186516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c8e800 0x557847ca6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 -- 172.21.15.37:0/1995186516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83f40 msgr2=0x557847c7eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83f40 0x557847c7eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 -- 172.21.15.37:0/1995186516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c8e800 msgr2=0x557847ca6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c8e800 0x557847ca6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 -- 172.21.15.37:0/1995186516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557847c80700 con 0x557847c83620 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c8e800 0x557847ca6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.507+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c9a290 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb5c409c200 tx=0x7fb5c409c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a290bf320fb08e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5b9ffb640 1 -- 172.21.15.37:0/1995186516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4020040 con 0x557847c83620 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5b9ffb640 1 -- 172.21.15.37:0/1995186516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c40a4e30 con 0x557847c83620 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5b9ffb640 1 -- 172.21.15.37:0/1995186516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40a49a0 con 0x557847c83620 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 -- 172.21.15.37:0/1995186516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 msgr2=0x557847c9a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c9a290 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb5c409c200 tx=0x7fb5c409c170 comp rx=0 tx=0).stop 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 -- 172.21.15.37:0/1995186516 shutdown_connections 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c8e800 0x557847ca6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c83f40 0x557847c7eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 --2- 172.21.15.37:0/1995186516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c9a290 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 -- 172.21.15.37:0/1995186516 >> 172.21.15.37:0/1995186516 conn(0x557847c7fb30 msgr2=0x557847c7e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 -- 172.21.15.37:0/1995186516 shutdown_connections 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.508+0000 7fb5dd815740 1 -- 172.21.15.37:0/1995186516 wait complete. 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 -- start start 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 Processor -- start 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c96b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c93f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c8e800 0x557847c94440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41ac2b0 con 0x557847c83620 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41ad720 con 0x557847c83f40 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d41ac980 con 0x557847c8e800 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5dd815740 10 client.0 _ll_register_callbacks cb 0x7fb5cc00f660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c96b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c96b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45610/0 (socket says 172.21.15.37:45610) 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5d3fff640 1 -- 172.21.15.37:0/557963943 learned_addr learned my addr 172.21.15.37:0/557963943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.509+0000 7fb5d8fc0640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c8e800 0x557847c94440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c93f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d3fff640 1 -- 172.21.15.37:0/557963943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c8e800 msgr2=0x557847c94440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c8e800 0x557847c94440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d3fff640 1 -- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 msgr2=0x557847c93f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c93f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d3fff640 1 -- 172.21.15.37:0/557963943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557847c94c20 con 0x557847c83620 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c93f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5d3fff640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c96b70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004770 tx=0x7fb5c40a06f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac5bca74cb35091 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5b37fe640 1 -- 172.21.15.37:0/557963943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4020050 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5b37fe640 1 -- 172.21.15.37:0/557963943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fb5c40cdbf0 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c40a4e30 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5b37fe640 1 -- 172.21.15.37:0/557963943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb5c40a49a0 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5dd815740 10 client.16647 fetch_fsmap learned FSMap version 41 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557847d04d60 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5b37fe640 1 -- 172.21.15.37:0/557963943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c40a4e30 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.510+0000 7fb5b37fe640 1 -- 172.21.15.37:0/557963943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7fb5c4081c30 con 0x557847c83620 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5dd815740 10 client.16647 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5dd815740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5dd815740 20 client.16647 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5dd815740 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847d05340 0x557847d07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5dd815740 4 client.16647 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557847c8bcf0 con 0x557847d05340 2024-04-23T18:30:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847d05340 0x557847d07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5b37fe640 10 client.16647 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.511+0000 7fb5d37fe640 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847d05340 0x557847d07760 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84496593cc342a5f server_cookie=ff22fccda77e5564 in_seq=0 out_seq=0 2024-04-23T18:30:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.542+0000 7fb5b37fe640 1 -- 172.21.15.37:0/557963943 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557847c8bcf0 con 0x557847d05340 2024-04-23T18:30:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5b37fe640 10 client.16647 handle_command_reply: tid=0 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 client.16647 shutdown 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 msgr2=0x557847c96b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c96b70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004770 tx=0x7fb5c40a06f0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 shutdown_connections 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x557847d05340 0x557847d07760 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557847c8e800 0x557847c94440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5b37fe640 0 client.16647 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557847c83f40 0x557847c93f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 --2- 172.21.15.37:0/557963943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557847c83620 0x557847c96b70 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 >> 172.21.15.37:0/557963943 conn(0x557847c7fb30 msgr2=0x557847c898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 shutdown_connections 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 1 -- 172.21.15.37:0/557963943 wait complete. 2024-04-23T18:30:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.543+0000 7fb5dd815740 20 client.16647 trim_cache size 0 max 16384 2024-04-23T18:30:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:30:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:30:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:30:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb5bc0a3dc0 msgr2=0x7fb5bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb5bc0a3dc0 0x7fb5bc0a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009370 tx=0x7fb5cc0096a0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 msgr2=0x7fb5d4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 0x7fb5d4198710 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400d770 tx=0x7fb5c400dc30 comp rx=0 tx=0).stop 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 shutdown_connections 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fb5bc0a3dc0 0x7fb5bc0a6280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5d410c5d0 0x7fb5d419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d4103b20 0x7fb5d4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 --2- 172.21.15.37:0/166947252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5d4103200 0x7fb5d4198710 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 >> 172.21.15.37:0/166947252 conn(0x7fb5d40771d0 msgr2=0x7fb5d40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 shutdown_connections 2024-04-23T18:30:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:21.544+0000 7fb5daa23640 1 -- 172.21.15.37:0/166947252 wait complete. 2024-04-23T18:30:21.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 482136 3.42 MB/sec execute 386 sec latency 1606.456 ms 2024-04-23T18:30:22.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:21 smithi172 ceph-mon[32456]: pgmap v309: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:21 smithi173 ceph-mon[32906]: pgmap v309: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:22.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:21 smithi037 ceph-mon[29404]: pgmap v309: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 691 KiB/s wr, 54 op/s 2024-04-23T18:30:22.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 483812 3.45 MB/sec execute 387 sec latency 21.231 ms 2024-04-23T18:30:23.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 485709 3.46 MB/sec execute 388 sec latency 137.708 ms 2024-04-23T18:30:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:24 smithi172 ceph-mon[32456]: pgmap v310: 97 pgs: 97 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 3.1 MiB/s wr, 114 op/s 2024-04-23T18:30:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:24 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:30:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:24 smithi173 ceph-mon[32906]: pgmap v310: 97 pgs: 97 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 3.1 MiB/s wr, 114 op/s 2024-04-23T18:30:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:24 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:30:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:24 smithi037 ceph-mon[29404]: pgmap v310: 97 pgs: 97 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 3.1 MiB/s wr, 114 op/s 2024-04-23T18:30:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:24 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:30:24.843 INFO:tasks.workunit.client.0.smithi037.stdout: 1 487482 3.48 MB/sec execute 389 sec latency 18.810 ms 2024-04-23T18:30:25.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 488453 3.48 MB/sec execute 390 sec latency 500.657 ms 2024-04-23T18:30:26.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:26 smithi172 ceph-mon[32456]: pgmap v311: 97 pgs: 97 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 2.7 MiB/s wr, 83 op/s 2024-04-23T18:30:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:26 smithi173 ceph-mon[32906]: pgmap v311: 97 pgs: 97 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 2.7 MiB/s wr, 83 op/s 2024-04-23T18:30:26.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:26 smithi037 ceph-mon[29404]: pgmap v311: 97 pgs: 97 active+clean; 320 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 2.7 MiB/s wr, 83 op/s 2024-04-23T18:30:26.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 488453 3.47 MB/sec execute 391 sec latency 1500.752 ms 2024-04-23T18:30:27.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 488453 3.46 MB/sec execute 392 sec latency 2500.870 ms 2024-04-23T18:30:28.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:28 smithi172 ceph-mon[32456]: pgmap v312: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 155 op/s 2024-04-23T18:30:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:28 smithi173 ceph-mon[32906]: pgmap v312: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 155 op/s 2024-04-23T18:30:28.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:28 smithi037 ceph-mon[29404]: pgmap v312: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 155 op/s 2024-04-23T18:30:28.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 489179 3.46 MB/sec execute 393 sec latency 3223.792 ms 2024-04-23T18:30:29.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 490891 3.48 MB/sec execute 394 sec latency 18.293 ms 2024-04-23T18:30:30.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:30 smithi172 ceph-mon[32456]: pgmap v313: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 139 op/s 2024-04-23T18:30:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:30 smithi173 ceph-mon[32906]: pgmap v313: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 139 op/s 2024-04-23T18:30:30.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:30 smithi037 ceph-mon[29404]: pgmap v313: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 139 op/s 2024-04-23T18:30:30.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 492000 3.49 MB/sec execute 395 sec latency 406.578 ms 2024-04-23T18:30:31.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 492000 3.48 MB/sec execute 396 sec latency 1406.669 ms 2024-04-23T18:30:32.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:32 smithi172 ceph-mon[32456]: pgmap v314: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 139 op/s 2024-04-23T18:30:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:32 smithi173 ceph-mon[32906]: pgmap v314: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 139 op/s 2024-04-23T18:30:32.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:32 smithi037 ceph-mon[29404]: pgmap v314: 97 pgs: 97 active+clean; 327 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 139 op/s 2024-04-23T18:30:32.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 492726 3.47 MB/sec execute 397 sec latency 2148.500 ms 2024-04-23T18:30:33.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 494404 3.49 MB/sec execute 398 sec latency 15.159 ms 2024-04-23T18:30:34.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:34 smithi172 ceph-mon[32456]: pgmap v315: 97 pgs: 97 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-04-23T18:30:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:34 smithi173 ceph-mon[32906]: pgmap v315: 97 pgs: 97 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-04-23T18:30:34.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:34 smithi037 ceph-mon[29404]: pgmap v315: 97 pgs: 97 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2024-04-23T18:30:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 -- 172.21.15.37:0/1125743871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10106190 msgr2=0x7f4e1010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1125743871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10106190 0x7f4e1010dd60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 -- 172.21.15.37:0/1125743871 shutdown_connections 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1125743871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10106190 0x7f4e1010dd60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1125743871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e10101ac0 0x7f4e10105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1125743871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10101110 0x7f4e101014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 -- 172.21.15.37:0/1125743871 >> 172.21.15.37:0/1125743871 conn(0x7f4e10077fd0 msgr2=0x7f4e100783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 -- 172.21.15.37:0/1125743871 shutdown_connections 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.634+0000 7f4e17afa640 1 -- 172.21.15.37:0/1125743871 wait complete. 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 Processor -- start 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 -- start start 2024-04-23T18:30:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 0x7f4e100fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10106190 0x7f4e100ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e100ff780 0x7f4e101a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e10111130 con 0x7f4e10101110 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e10110fb0 con 0x7f4e10106190 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e17afa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101112b0 con 0x7f4e100ff780 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 0x7f4e100fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 0x7f4e100fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53898/0 (socket says 172.21.15.37:53898) 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e15896640 1 -- 172.21.15.37:0/1262827096 learned_addr learned my addr 172.21.15.37:0/1262827096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e16097640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e100ff780 0x7f4e101a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.635+0000 7f4e15095640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10106190 0x7f4e100ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15896640 1 -- 172.21.15.37:0/1262827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e100ff780 msgr2=0x7f4e101a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15896640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e100ff780 0x7f4e101a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15896640 1 -- 172.21.15.37:0/1262827096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10106190 msgr2=0x7f4e100ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15896640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10106190 0x7f4e100ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15896640 1 -- 172.21.15.37:0/1262827096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e101a6ef0 con 0x7f4e10101110 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15095640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10106190 0x7f4e100ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e15896640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 0x7f4e100fed00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00eab0 tx=0x7f4e0c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e06ffd640 1 -- 172.21.15.37:0/1262827096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00ce40 con 0x7f4e10101110 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e06ffd640 1 -- 172.21.15.37:0/1262827096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e0c011920 con 0x7f4e10101110 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e101a71e0 con 0x7f4e10101110 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e06ffd640 1 -- 172.21.15.37:0/1262827096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c010a30 con 0x7f4e10101110 2024-04-23T18:30:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.636+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e101a7730 con 0x7f4e10101110 2024-04-23T18:30:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.639+0000 7f4e1a814740 1 Processor -- start 2024-04-23T18:30:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.639+0000 7f4e06ffd640 1 -- 172.21.15.37:0/1262827096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e0c010bd0 con 0x7f4e10101110 2024-04-23T18:30:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.639+0000 7f4e1a814740 1 -- start start 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.639+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.639+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 0x55db265d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265d3800 0x55db265d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e1a814740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db2652d970 con 0x55db265c8f40 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e1a814740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55db264ca4a0 con 0x55db265c8620 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e1a814740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55db26529930 con 0x55db265d3800 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 0x55db265d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 0x55db265d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53900/0 (socket says 172.21.15.37:53900) 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 -- 172.21.15.37:0/6030672 learned_addr learned my addr 172.21.15.37:0/6030672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15095640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265d3800 0x55db265d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e16097640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e06ffd640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4dfc0a3df0 0x7f4dfc0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 -- 172.21.15.37:0/6030672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265d3800 msgr2=0x55db265d5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265d3800 0x55db265d5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 -- 172.21.15.37:0/6030672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 msgr2=0x55db265c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 -- 172.21.15.37:0/6030672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db26240210 con 0x55db265c8f40 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15095640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265d3800 0x55db265d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e16097640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e16097640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4dfc0a3df0 0x7f4dfc0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e06ffd640 1 -- 172.21.15.37:0/1262827096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f4e0c0d0e90 con 0x7f4e10101110 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.640+0000 7f4e15896640 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 0x55db265d32c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0ce480 tx=0x7f4e0c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ae9e387d7497f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4dfbfff640 1 -- 172.21.15.37:0/6030672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c09c6f0 con 0x55db265c8f40 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e16097640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4dfc0a3df0 0x7f4dfc0a62b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e08006010 tx=0x7f4e08009260 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4dfbfff640 1 -- 172.21.15.37:0/6030672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e0c09d490 con 0x55db265c8f40 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4dfbfff640 1 -- 172.21.15.37:0/6030672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0d4a30 con 0x55db265c8f40 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 -- 172.21.15.37:0/6030672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 msgr2=0x55db265d32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 0x55db265d32c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0ce480 tx=0x7f4e0c021000 comp rx=0 tx=0).stop 2024-04-23T18:30:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 -- 172.21.15.37:0/6030672 shutdown_connections 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265d3800 0x55db265d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8f40 0x55db265d32c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 --2- 172.21.15.37:0/6030672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.641+0000 7f4e1a814740 1 -- 172.21.15.37:0/6030672 >> 172.21.15.37:0/6030672 conn(0x55db265c4b30 msgr2=0x55db265c4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 -- 172.21.15.37:0/6030672 shutdown_connections 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 -- 172.21.15.37:0/6030672 wait complete. 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 -- start start 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 Processor -- start 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55db265d6370 con 0x55db265c8620 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55db264c80b0 con 0x55db265d3800 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55db265d64f0 con 0x55db265c8f40 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e1a814740 10 client.0 _ll_register_callbacks cb 0x55db264b5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53914/0 (socket says 172.21.15.37:53914) 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e15896640 1 -- 172.21.15.37:0/175204513 learned_addr learned my addr 172.21.15.37:0/175204513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.642+0000 7f4e15095640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e16097640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e15896640 1 -- 172.21.15.37:0/175204513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 msgr2=0x55db265d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e15896640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e15896640 1 -- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 msgr2=0x55db265d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e15896640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e15896640 1 -- 172.21.15.37:0/175204513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2630c620 con 0x55db265c8620 2024-04-23T18:30:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e16097640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e15896640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265d3190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c09ca90 tx=0x7f4e0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb615a685b6b5f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4df97fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4df97fa640 1 -- 172.21.15.37:0/175204513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c01c0c0 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4df97fa640 1 -- 172.21.15.37:0/175204513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f4e0c099400 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e0c09d490 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4df97fa640 1 -- 172.21.15.37:0/175204513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4e0c0d4a30 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e1a814740 10 client.16659 fetch_fsmap learned FSMap version 42 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55db265e22c0 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.643+0000 7f4df97fa640 1 -- 172.21.15.37:0/175204513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e0c09d490 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4df97fa640 1 -- 172.21.15.37:0/175204513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7f4e0c00f040 con 0x55db265c8620 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e1a814740 10 client.16659 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e1a814740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e1a814740 20 client.16659 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e1a814740 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db265e3610 0x55db265e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e1a814740 4 client.16659 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db2653b390 con 0x55db265e3610 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e15095640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db265e3610 0x55db265e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4df97fa640 10 client.16659 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.644+0000 7f4e15095640 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db265e3610 0x55db265e5a00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53437a84993493ce server_cookie=527f4371f84e302c in_seq=0 out_seq=0 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4df97fa640 1 -- 172.21.15.37:0/175204513 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db2653b390 con 0x55db265e3610 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4df97fa640 10 client.16659 handle_command_reply: tid=0 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 client.16659 shutdown 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 msgr2=0x55db265d3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265d3190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c09ca90 tx=0x7f4e0c00c830 comp rx=0 tx=0).stop 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 shutdown_connections 2024-04-23T18:30:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db265e3610 0x55db265e5a00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 --2- 172.21.15.37:0/175204513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265d3190 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4df97fa640 0 client.16659 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 >> 172.21.15.37:0/175204513 conn(0x55db265c4b30 msgr2=0x55db265ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 shutdown_connections 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 1 -- 172.21.15.37:0/175204513 wait complete. 2024-04-23T18:30:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.645+0000 7f4e1a814740 20 client.16659 trim_cache size 0 max 16384 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 Processor -- start 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 -- start start 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101b2760 con 0x55db265c8620 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101affd0 con 0x55db265d3800 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e1a814740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101ade40 con 0x55db265c8f40 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e15896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265df230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52200/0 (socket says 172.21.15.37:52200) 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.660+0000 7f4e15896640 1 -- 172.21.15.37:0/1294255723 learned_addr learned my addr 172.21.15.37:0/1294255723 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e16097640 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15095640 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15896640 1 -- 172.21.15.37:0/1294255723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 msgr2=0x55db265c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15896640 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15896640 1 -- 172.21.15.37:0/1294255723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 msgr2=0x55db265d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15896640 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15896640 1 -- 172.21.15.37:0/1294255723 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db265c5700 con 0x55db265c8620 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15095640 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e15896640 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265df230 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0991f0 tx=0x7f4e0c09ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2c09c901bf7f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4dfb7fe640 1 -- 172.21.15.37:0/1294255723 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c09c6f0 con 0x55db265c8620 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4dfb7fe640 1 -- 172.21.15.37:0/1294255723 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e0c0ba020 con 0x55db265c8620 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4dfb7fe640 1 -- 172.21.15.37:0/1294255723 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0ce810 con 0x55db265c8620 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e1a814740 1 -- 172.21.15.37:0/1294255723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 msgr2=0x55db265df230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.661+0000 7f4e1a814740 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265df230 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0991f0 tx=0x7f4e0c09ce10 comp rx=0 tx=0).stop 2024-04-23T18:30:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- 172.21.15.37:0/1294255723 shutdown_connections 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265d3800 0x55db265d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 --2- 172.21.15.37:0/1294255723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265c8620 0x55db265df230 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- 172.21.15.37:0/1294255723 >> 172.21.15.37:0/1294255723 conn(0x55db265c4b30 msgr2=0x55db265c3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- 172.21.15.37:0/1294255723 shutdown_connections 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- 172.21.15.37:0/1294255723 wait complete. 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- start start 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 Processor -- start 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 0x55db265d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101ae490 con 0x55db265d3800 2024-04-23T18:30:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101af900 con 0x55db265c8620 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101aeb60 con 0x55db265c8f40 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e1a814740 10 client.0 _ll_register_callbacks cb 0x7f4e0c098ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e16097640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 0x55db265d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e16097640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 0x55db265d7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52202/0 (socket says 172.21.15.37:52202) 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.662+0000 7f4e16097640 1 -- 172.21.15.37:0/624796245 learned_addr learned my addr 172.21.15.37:0/624796245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e15095640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e15896640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e16097640 1 -- 172.21.15.37:0/624796245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 msgr2=0x55db265d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e16097640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e16097640 1 -- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 msgr2=0x55db265db8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e16097640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265db8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e16097640 1 -- 172.21.15.37:0/624796245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db265d7800 con 0x55db265d3800 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e16097640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 0x55db265d7240 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c4013b814a78c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4df97fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4df97fa640 1 -- 172.21.15.37:0/624796245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08035040 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4df97fa640 1 -- 172.21.15.37:0/624796245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f4e080314a0 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e0c0ba020 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4df97fa640 1 -- 172.21.15.37:0/624796245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4e080316a0 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e1a814740 10 client.16665 fetch_fsmap learned FSMap version 42 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55db26649610 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.663+0000 7f4df97fa640 1 -- 172.21.15.37:0/624796245 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e0c0ba020 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4df97fa640 1 -- 172.21.15.37:0/624796245 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7f4e0805b990 con 0x55db265d3800 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e1a814740 10 client.16665 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e1a814740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e1a814740 20 client.16665 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e1a814740 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db26649df0 0x55db2664c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e1a814740 4 client.16665 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db26535200 con 0x55db26649df0 2024-04-23T18:30:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e15896640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db26649df0 0x55db2664c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4e15896640 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db26649df0 0x55db2664c210 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23320b68c84e6d1b server_cookie=e573f13da7f9f1c6 in_seq=0 out_seq=0 2024-04-23T18:30:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.664+0000 7f4df97fa640 10 client.16665 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4df97fa640 1 -- 172.21.15.37:0/624796245 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db26535200 con 0x55db26649df0 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4df97fa640 10 client.16665 handle_command_reply: tid=0 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 client.16665 shutdown 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 msgr2=0x55db265d7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 0x55db265d7240 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 shutdown_connections 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55db26649df0 0x55db2664c210 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db265d3800 0x55db265d7240 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55db265c8f40 0x55db265d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 --2- 172.21.15.37:0/624796245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db265c8620 0x55db265db8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4df97fa640 0 client.16665 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 >> 172.21.15.37:0/624796245 conn(0x55db265c4b30 msgr2=0x55db265c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.704+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 shutdown_connections 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e1a814740 1 -- 172.21.15.37:0/624796245 wait complete. 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e1a814740 20 client.16665 trim_cache size 0 max 16384 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:30:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:30:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:30:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4dfc0a3df0 msgr2=0x7f4dfc0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4dfc0a3df0 0x7f4dfc0a62b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e08006010 tx=0x7f4e08009260 comp rx=0 tx=0).stop 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 msgr2=0x7f4e100fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 0x7f4e100fed00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00eab0 tx=0x7f4e0c00ef70 comp rx=0 tx=0).stop 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 shutdown_connections 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4dfc0a3df0 0x7f4dfc0a62b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e100ff780 0x7f4e101a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e10106190 0x7f4e100ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 --2- 172.21.15.37:0/1262827096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e10101110 0x7f4e100fed00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 >> 172.21.15.37:0/1262827096 conn(0x7f4e10077fd0 msgr2=0x7f4e1006a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 shutdown_connections 2024-04-23T18:30:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:34.705+0000 7f4e17afa640 1 -- 172.21.15.37:0/1262827096 wait complete. 2024-04-23T18:30:34.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 496052 3.50 MB/sec execute 399 sec latency 215.558 ms 2024-04-23T18:30:35.844 INFO:tasks.workunit.client.0.smithi037.stdout: 1 497845 3.52 MB/sec execute 400 sec latency 15.171 ms 2024-04-23T18:30:36.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:36 smithi172 ceph-mon[32456]: pgmap v316: 97 pgs: 97 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-04-23T18:30:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:36 smithi173 ceph-mon[32906]: pgmap v316: 97 pgs: 97 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-04-23T18:30:36.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:36 smithi037 ceph-mon[29404]: pgmap v316: 97 pgs: 97 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-04-23T18:30:36.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 499092 3.53 MB/sec execute 401 sec latency 286.639 ms 2024-04-23T18:30:37.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 499092 3.52 MB/sec execute 402 sec latency 1286.715 ms 2024-04-23T18:30:38.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:38 smithi172 ceph-mon[32456]: pgmap v317: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-23T18:30:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:38 smithi173 ceph-mon[32906]: pgmap v317: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-23T18:30:38.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:38 smithi037 ceph-mon[29404]: pgmap v317: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-23T18:30:38.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 499832 3.52 MB/sec execute 403 sec latency 2000.232 ms 2024-04-23T18:30:39.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 501541 3.54 MB/sec execute 404 sec latency 14.567 ms 2024-04-23T18:30:40.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:40 smithi172 ceph-mon[32456]: pgmap v318: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:30:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:40 smithi173 ceph-mon[32906]: pgmap v318: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:30:40.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:40 smithi037 ceph-mon[29404]: pgmap v318: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:30:40.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 502633 3.54 MB/sec execute 405 sec latency 406.760 ms 2024-04-23T18:30:41.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 502633 3.53 MB/sec execute 406 sec latency 1406.878 ms 2024-04-23T18:30:42.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:42 smithi172 ceph-mon[32456]: pgmap v319: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:30:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:42 smithi173 ceph-mon[32906]: pgmap v319: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:30:42.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:42 smithi037 ceph-mon[29404]: pgmap v319: 97 pgs: 97 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:30:42.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 503390 3.53 MB/sec execute 407 sec latency 2137.052 ms 2024-04-23T18:30:43.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 505111 3.55 MB/sec execute 408 sec latency 11.914 ms 2024-04-23T18:30:44.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:44 smithi172 ceph-mon[32456]: pgmap v320: 97 pgs: 97 active+clean; 344 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-04-23T18:30:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:44 smithi173 ceph-mon[32906]: pgmap v320: 97 pgs: 97 active+clean; 344 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-04-23T18:30:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:44 smithi037 ceph-mon[29404]: pgmap v320: 97 pgs: 97 active+clean; 344 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-04-23T18:30:44.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 506183 3.55 MB/sec execute 409 sec latency 419.542 ms 2024-04-23T18:30:45.300 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:45 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:45.305 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:45 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:45.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:45 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:45.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 506183 3.54 MB/sec execute 410 sec latency 1419.582 ms 2024-04-23T18:30:46.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: pgmap v321: 97 pgs: 97 active+clean; 344 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-04-23T18:30:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:46 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: pgmap v321: 97 pgs: 97 active+clean; 344 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:46 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: pgmap v321: 97 pgs: 97 active+clean; 344 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-04-23T18:30:46.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:46 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:46.845 INFO:tasks.workunit.client.0.smithi037.stdout: 1 506183 3.54 MB/sec execute 411 sec latency 2419.656 ms 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:47.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:47.338 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:47.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.356 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.397 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.397 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.397 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:47.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:47.398 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:47.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:30:47.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:47.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:47.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:30:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.804+0000 7f17df77d640 1 -- 172.21.15.37:0/612165125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d81016b0 msgr2=0x7f17d810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.804+0000 7f17df77d640 1 --2- 172.21.15.37:0/612165125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d81016b0 0x7f17d810e370 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009900 tx=0x7f17c802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 -- 172.21.15.37:0/612165125 shutdown_connections 2024-04-23T18:30:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 --2- 172.21.15.37:0/612165125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d810e8b0 0x7f17d8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 --2- 172.21.15.37:0/612165125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d81016b0 0x7f17d810e370 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 --2- 172.21.15.37:0/612165125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d8100d00 0x7f17d81010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 -- 172.21.15.37:0/612165125 >> 172.21.15.37:0/612165125 conn(0x7f17d8077fd0 msgr2=0x7f17d80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 -- 172.21.15.37:0/612165125 shutdown_connections 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 -- 172.21.15.37:0/612165125 wait complete. 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 Processor -- start 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.805+0000 7f17df77d640 1 -- start start 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17df77d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 0x7f17d81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17df77d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d81016b0 0x7f17d81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17df77d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d810e8b0 0x7f17d819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17df77d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8114400 con 0x7f17d8100d00 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17df77d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8114280 con 0x7f17d81016b0 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17df77d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8114580 con 0x7f17d810e8b0 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 0x7f17d81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 0x7f17d81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35520/0 (socket says 172.21.15.37:35520) 2024-04-23T18:30:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 -- 172.21.15.37:0/340704456 learned_addr learned my addr 172.21.15.37:0/340704456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d810e8b0 0x7f17d819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dcd18640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d81016b0 0x7f17d81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 -- 172.21.15.37:0/340704456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d810e8b0 msgr2=0x7f17d819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d810e8b0 0x7f17d819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 -- 172.21.15.37:0/340704456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d81016b0 msgr2=0x7f17d81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d81016b0 0x7f17d81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 -- 172.21.15.37:0/340704456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d819afb0 con 0x7f17d8100d00 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dcd18640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d81016b0 0x7f17d81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d810e8b0 0x7f17d819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17dd519640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 0x7f17d81a0620 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f17d400d770 tx=0x7f17d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.806+0000 7f17ce7fc640 1 -- 172.21.15.37:0/340704456 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4017070 con 0x7f17d8100d00 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.807+0000 7f17ce7fc640 1 -- 172.21.15.37:0/340704456 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d40054d0 con 0x7f17d8100d00 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.807+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d819b2a0 con 0x7f17d8100d00 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.807+0000 7f17ce7fc640 1 -- 172.21.15.37:0/340704456 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4004c30 con 0x7f17d8100d00 2024-04-23T18:30:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.807+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81a7470 con 0x7f17d8100d00 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17ce7fc640 1 -- 172.21.15.37:0/340704456 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17d4004e10 con 0x7f17d8100d00 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 Processor -- start 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 -- start start 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f11493a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493f40 0x563f1149e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 0x563f114a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f113f8970 con 0x563f1149e800 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f113954a0 con 0x563f11493620 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17e2572740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563f113f4930 con 0x563f11493f40 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 0x563f114a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.810+0000 7f17ce7fc640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f17c40a3dc0 0x7f17c40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 0x563f114a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35532/0 (socket says 172.21.15.37:35532) 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 -- 172.21.15.37:0/4160725905 learned_addr learned my addr 172.21.15.37:0/4160725905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f11493a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dd519640 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493f40 0x563f1149e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f17c40a3dc0 0x7f17c40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17ce7fc640 1 -- 172.21.15.37:0/340704456 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f17d40d03c0 con 0x7f17d8100d00 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 -- 172.21.15.37:0/4160725905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493f40 msgr2=0x563f1149e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493f40 0x563f1149e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 -- 172.21.15.37:0/4160725905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 msgr2=0x563f11493a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f11493a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 -- 172.21.15.37:0/4160725905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f1110b210 con 0x563f1149e800 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17dcd18640 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 0x563f114a0bf0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009900 tx=0x7f17c8032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd18079874e03f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f17c40a3dc0 0x7f17c40a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005c40 tx=0x7f17d0009280 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17c37fe640 1 -- 172.21.15.37:0/4160725905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c803f070 con 0x563f1149e800 2024-04-23T18:30:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17c37fe640 1 -- 172.21.15.37:0/4160725905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c80316a0 con 0x563f1149e800 2024-04-23T18:30:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.811+0000 7f17c37fe640 1 -- 172.21.15.37:0/4160725905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8035070 con 0x563f1149e800 2024-04-23T18:30:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 -- 172.21.15.37:0/4160725905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 msgr2=0x563f114a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 0x563f114a0bf0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009900 tx=0x7f17c8032cd0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 -- 172.21.15.37:0/4160725905 shutdown_connections 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f1149e800 0x563f114a0bf0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493f40 0x563f1149e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 --2- 172.21.15.37:0/4160725905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f11493a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 -- 172.21.15.37:0/4160725905 >> 172.21.15.37:0/4160725905 conn(0x563f1148fb30 msgr2=0x563f1148ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 -- 172.21.15.37:0/4160725905 shutdown_connections 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.812+0000 7f17e2572740 1 -- 172.21.15.37:0/4160725905 wait complete. 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 -- start start 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 Processor -- start 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f1149e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f1149b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f1149ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f114a13d0 con 0x563f11493f40 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f113930b0 con 0x563f1149e800 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563f114a1550 con 0x563f11493620 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17e2572740 10 client.0 _ll_register_callbacks cb 0x563f11380d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f1149b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f1149b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35536/0 (socket says 172.21.15.37:35536) 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17dcd18640 1 -- 172.21.15.37:0/3128224380 learned_addr learned my addr 172.21.15.37:0/3128224380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.813+0000 7f17dd519640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f1149e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f1149ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dcd18640 1 -- 172.21.15.37:0/3128224380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 msgr2=0x563f1149e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dcd18640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f1149e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dcd18640 1 -- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 msgr2=0x563f1149ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dcd18640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f1149ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dcd18640 1 -- 172.21.15.37:0/3128224380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f111d7620 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dd519640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f1149e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17dcd18640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f1149b490 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006010 tx=0x7f17c80045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366c145b09a92e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17c0ff9640 1 -- 172.21.15.37:0/3128224380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c804a030 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17c0ff9640 1 -- 172.21.15.37:0/3128224380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f17c80314a0 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c80316a0 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17c0ff9640 1 -- 172.21.15.37:0/3128224380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f17c8054050 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17e2572740 10 client.16677 fetch_fsmap learned FSMap version 42 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563f114ad340 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.814+0000 7f17c0ff9640 1 -- 172.21.15.37:0/3128224380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c80316a0 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17c0ff9640 1 -- 172.21.15.37:0/3128224380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7f17c8004750 con 0x563f11493f40 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17e2572740 10 client.16677 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17e2572740 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17e2572740 20 client.16677 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17e2572740 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f114ae690 0x563f114b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17e2572740 4 client.16677 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f11406390 con 0x563f114ae690 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.815+0000 7f17dd519640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f114ae690 0x563f114b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17dd519640 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f114ae690 0x563f114b0a80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee76cb90b8859002 server_cookie=29a5bc59594aaabe in_seq=0 out_seq=0 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17c0ff9640 10 client.16677 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17c0ff9640 1 -- 172.21.15.37:0/3128224380 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f11406390 con 0x563f114ae690 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17c0ff9640 10 client.16677 handle_command_reply: tid=0 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17e2572740 1 client.16677 shutdown 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 msgr2=0x563f1149b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.816+0000 7f17e2572740 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f1149b490 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006010 tx=0x7f17c80045b0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 shutdown_connections 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f114ae690 0x563f114b0a80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f1149ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f1149b490 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 --2- 172.21.15.37:0/3128224380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f1149e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17c0ff9640 0 client.16677 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 >> 172.21.15.37:0/3128224380 conn(0x563f1148fb30 msgr2=0x563f11495860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 shutdown_connections 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 1 -- 172.21.15.37:0/3128224380 wait complete. 2024-04-23T18:30:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.817+0000 7f17e2572740 20 client.16677 trim_cache size 0 max 16384 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 Processor -- start 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 -- start start 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f114aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f1149e800 0x563f114b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17d81ae340 con 0x563f11493f40 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d81abbb0 con 0x563f11493620 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17e2572740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f17d81a9a20 con 0x563f1149e800 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35542/0 (socket says 172.21.15.37:35542) 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 -- 172.21.15.37:0/3228560159 learned_addr learned my addr 172.21.15.37:0/3228560159 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f1149e800 0x563f114b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dd519640 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f114aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 -- 172.21.15.37:0/3228560159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f1149e800 msgr2=0x563f114b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f1149e800 0x563f114b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 -- 172.21.15.37:0/3228560159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 msgr2=0x563f114aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f114aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dcd18640 1 -- 172.21.15.37:0/3228560159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f1148b500 con 0x563f11493f40 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.832+0000 7f17dd519640 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f114aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17dcd18640 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114b60e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004ce0 tx=0x7f17c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d8ce674e4c0d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17c2ffd640 1 -- 172.21.15.37:0/3228560159 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c804a070 con 0x563f11493f40 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17c2ffd640 1 -- 172.21.15.37:0/3228560159 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c80314a0 con 0x563f11493f40 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17c2ffd640 1 -- 172.21.15.37:0/3228560159 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c803f070 con 0x563f11493f40 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 -- 172.21.15.37:0/3228560159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 msgr2=0x563f114b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114b60e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004ce0 tx=0x7f17c8000bc0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 -- 172.21.15.37:0/3228560159 shutdown_connections 2024-04-23T18:30:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f1149e800 0x563f114b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114b60e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 --2- 172.21.15.37:0/3228560159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f11493620 0x563f114aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 -- 172.21.15.37:0/3228560159 >> 172.21.15.37:0/3228560159 conn(0x563f1148fb30 msgr2=0x563f11499d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 -- 172.21.15.37:0/3228560159 shutdown_connections 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.833+0000 7f17e2572740 1 -- 172.21.15.37:0/3228560159 wait complete. 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 -- start start 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 Processor -- start 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f114a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f114a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17d81aa070 con 0x563f11493f40 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d81ab4e0 con 0x563f1149e800 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f17d81aa740 con 0x563f11493620 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17e2572740 10 client.0 _ll_register_callbacks cb 0x7f17c8004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17dcd18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35556/0 (socket says 172.21.15.37:35556) 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17dcd18640 1 -- 172.21.15.37:0/2458485284 learned_addr learned my addr 172.21.15.37:0/2458485284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.834+0000 7f17dd519640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f114a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f114a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17dcd18640 1 -- 172.21.15.37:0/2458485284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 msgr2=0x563f114a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17dcd18640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f114a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17dcd18640 1 -- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 msgr2=0x563f114a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17dcd18640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f114a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17dcd18640 1 -- 172.21.15.37:0/2458485284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f11513b30 con 0x563f11493f40 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17ddd1a640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f114a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17dcd18640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114a48b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006010 tx=0x7f17c8032af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77cdf5f3069dda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17c0ff9640 1 -- 172.21.15.37:0/2458485284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c803f070 con 0x563f11493f40 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17c0ff9640 1 -- 172.21.15.37:0/2458485284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f17c8004f60 con 0x563f11493f40 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c80314a0 con 0x563f11493f40 2024-04-23T18:30:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17c0ff9640 1 -- 172.21.15.37:0/2458485284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f17c804a040 con 0x563f11493f40 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17e2572740 10 client.16683 fetch_fsmap learned FSMap version 42 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563f11514d70 con 0x563f11493f40 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.835+0000 7f17c0ff9640 1 -- 172.21.15.37:0/2458485284 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c80314a0 con 0x563f11493f40 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17c0ff9640 1 -- 172.21.15.37:0/2458485284 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7f17c80644c0 con 0x563f11493f40 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17e2572740 10 client.16683 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17e2572740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17e2572740 20 client.16683 populate_metadata read hostname 'smithi037' 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17e2572740 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f115154c0 0x563f115178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17e2572740 4 client.16683 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f11400200 con 0x563f115154c0 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17dd519640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f115154c0 0x563f115178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17c0ff9640 10 client.16683 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.836+0000 7f17dd519640 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f115154c0 0x563f115178e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e78adee4ffa23a server_cookie=abd730b94a257dee in_seq=0 out_seq=0 2024-04-23T18:30:47.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 506186 3.53 MB/sec execute 412 sec latency 3418.869 ms 2024-04-23T18:30:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17c0ff9640 1 -- 172.21.15.37:0/2458485284 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f11400200 con 0x563f115154c0 2024-04-23T18:30:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17c0ff9640 10 client.16683 handle_command_reply: tid=0 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 client.16683 shutdown 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 msgr2=0x563f114a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114a48b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006010 tx=0x7f17c8032af0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 shutdown_connections 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x563f115154c0 0x563f115178e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f1149e800 0x563f114a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f11493f40 0x563f114a48b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17c0ff9640 0 client.16683 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 --2- 172.21.15.37:0/2458485284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563f11493620 0x563f114a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 >> 172.21.15.37:0/2458485284 conn(0x563f1148fb30 msgr2=0x563f114a9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 shutdown_connections 2024-04-23T18:30:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 1 -- 172.21.15.37:0/2458485284 wait complete. 2024-04-23T18:30:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.873+0000 7f17e2572740 20 client.16683 trim_cache size 0 max 16384 2024-04-23T18:30:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:30:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:30:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:30:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:30:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f17c40a3dc0 msgr2=0x7f17c40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f17c40a3dc0 0x7f17c40a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005c40 tx=0x7f17d0009280 comp rx=0 tx=0).stop 2024-04-23T18:30:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 msgr2=0x7f17d81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 0x7f17d81a0620 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f17d400d770 tx=0x7f17d400dc30 comp rx=0 tx=0).stop 2024-04-23T18:30:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 shutdown_connections 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f17c40a3dc0 0x7f17c40a6280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17d810e8b0 0x7f17d819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d81016b0 0x7f17d81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 --2- 172.21.15.37:0/340704456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8100d00 0x7f17d81a0620 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 >> 172.21.15.37:0/340704456 conn(0x7f17d8077fd0 msgr2=0x7f17d810fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 shutdown_connections 2024-04-23T18:30:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:30:47.874+0000 7f17df77d640 1 -- 172.21.15.37:0/340704456 wait complete. 2024-04-23T18:30:48.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-23T18:30:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-23T18:30:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T18:30:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T18:30:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-23T18:30:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: Unable to set osd_memory_target on smithi037 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-23T18:30:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:48 smithi172 ceph-mon[32456]: pgmap v322: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: Unable to set osd_memory_target on smithi037 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-23T18:30:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:48 smithi173 ceph-mon[32906]: pgmap v322: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-23T18:30:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-23T18:30:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-23T18:30:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T18:30:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T18:30:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-23T18:30:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: Unable to set osd_memory_target on smithi037 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-04-23T18:30:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:48 smithi037 ceph-mon[29404]: pgmap v322: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-23T18:30:48.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 508395 3.55 MB/sec execute 413 sec latency 30.875 ms 2024-04-23T18:30:49.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 509726 3.56 MB/sec execute 414 sec latency 169.859 ms 2024-04-23T18:30:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:50 smithi172 ceph-mon[32456]: pgmap v323: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.0 MiB/s wr, 84 op/s 2024-04-23T18:30:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:50 smithi173 ceph-mon[32906]: pgmap v323: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.0 MiB/s wr, 84 op/s 2024-04-23T18:30:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:50 smithi037 ceph-mon[29404]: pgmap v323: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.0 MiB/s wr, 84 op/s 2024-04-23T18:30:50.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 509726 3.55 MB/sec execute 415 sec latency 1169.902 ms 2024-04-23T18:30:51.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 509726 3.54 MB/sec execute 416 sec latency 2170.011 ms 2024-04-23T18:30:52.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:52 smithi172 ceph-mon[32456]: pgmap v324: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.0 MiB/s wr, 84 op/s 2024-04-23T18:30:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:52 smithi173 ceph-mon[32906]: pgmap v324: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.0 MiB/s wr, 84 op/s 2024-04-23T18:30:52.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:52 smithi037 ceph-mon[29404]: pgmap v324: 97 pgs: 97 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.0 MiB/s wr, 84 op/s 2024-04-23T18:30:52.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 510464 3.54 MB/sec execute 417 sec latency 2897.588 ms 2024-04-23T18:30:53.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 512056 3.56 MB/sec execute 418 sec latency 28.896 ms 2024-04-23T18:30:54.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:54 smithi172 ceph-mon[32456]: pgmap v325: 97 pgs: 97 active+clean; 352 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-04-23T18:30:54.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:54 smithi173 ceph-mon[32906]: pgmap v325: 97 pgs: 97 active+clean; 352 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-04-23T18:30:54.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:54 smithi037 ceph-mon[29404]: pgmap v325: 97 pgs: 97 active+clean; 352 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-04-23T18:30:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 513272 3.56 MB/sec execute 419 sec latency 277.462 ms 2024-04-23T18:30:55.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 513272 3.55 MB/sec execute 420 sec latency 1277.611 ms 2024-04-23T18:30:56.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:56 smithi172 ceph-mon[32456]: pgmap v326: 97 pgs: 97 active+clean; 352 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-04-23T18:30:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:56 smithi173 ceph-mon[32906]: pgmap v326: 97 pgs: 97 active+clean; 352 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-04-23T18:30:56.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:56 smithi037 ceph-mon[29404]: pgmap v326: 97 pgs: 97 active+clean; 352 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2024-04-23T18:30:56.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 513272 3.54 MB/sec execute 421 sec latency 2277.684 ms 2024-04-23T18:30:57.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 513272 3.53 MB/sec execute 422 sec latency 3277.794 ms 2024-04-23T18:30:58.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:30:58 smithi172 ceph-mon[32456]: pgmap v327: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-04-23T18:30:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:30:58 smithi173 ceph-mon[32906]: pgmap v327: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-04-23T18:30:58.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:30:58 smithi037 ceph-mon[29404]: pgmap v327: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-04-23T18:30:58.846 INFO:tasks.workunit.client.0.smithi037.stdout: 1 514029 3.53 MB/sec execute 423 sec latency 3991.291 ms 2024-04-23T18:30:59.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 515786 3.55 MB/sec execute 424 sec latency 11.526 ms 2024-04-23T18:31:00.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:00 smithi172 ceph-mon[32456]: pgmap v328: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-23T18:31:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:00 smithi173 ceph-mon[32906]: pgmap v328: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-23T18:31:00.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:00 smithi037 ceph-mon[29404]: pgmap v328: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-23T18:31:00.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 516816 3.55 MB/sec execute 425 sec latency 401.472 ms 2024-04-23T18:31:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 -- 172.21.15.37:0/3439854468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c101890 msgr2=0x7f032c105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 --2- 172.21.15.37:0/3439854468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c101890 0x7f032c105d50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0320009930 tx=0x7f032002a4b0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 -- 172.21.15.37:0/3439854468 shutdown_connections 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 --2- 172.21.15.37:0/3439854468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c106370 0x7f032c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 --2- 172.21.15.37:0/3439854468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c101890 0x7f032c105d50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 --2- 172.21.15.37:0/3439854468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f032c100f70 0x7f032c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.965+0000 7f0333d76640 1 -- 172.21.15.37:0/3439854468 >> 172.21.15.37:0/3439854468 conn(0x7f032c077fd0 msgr2=0x7f032c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 -- 172.21.15.37:0/3439854468 shutdown_connections 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 -- 172.21.15.37:0/3439854468 wait complete. 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 Processor -- start 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 -- start start 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f032c100f70 0x7f032c1a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c101890 0x7f032c1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 0x7f032c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1187c0 con 0x7f032c106370 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f032c118640 con 0x7f032c101890 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.966+0000 7f0333d76640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f032c118940 con 0x7f032c100f70 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 0x7f032c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 0x7f032c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57218/0 (socket says 172.21.15.37:57218) 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 -- 172.21.15.37:0/88286043 learned_addr learned my addr 172.21.15.37:0/88286043 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0331311640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c101890 0x7f032c1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0331b12640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f032c100f70 0x7f032c1a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 -- 172.21.15.37:0/88286043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f032c100f70 msgr2=0x7f032c1a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f032c100f70 0x7f032c1a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 -- 172.21.15.37:0/88286043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c101890 msgr2=0x7f032c1a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c101890 0x7f032c1a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 -- 172.21.15.37:0/88286043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c19f240 con 0x7f032c106370 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0331311640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c101890 0x7f032c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0332313640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 0x7f032c19eac0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f031c00cad0 tx=0x7f031c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f032affd640 1 -- 172.21.15.37:0/88286043 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00ae20 con 0x7f032c106370 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f032affd640 1 -- 172.21.15.37:0/88286043 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c005020 con 0x7f032c106370 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c19f530 con 0x7f032c106370 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f032affd640 1 -- 172.21.15.37:0/88286043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00f6b0 con 0x7f032c106370 2024-04-23T18:31:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.967+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1ab7d0 con 0x7f032c106370 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f032affd640 1 -- 172.21.15.37:0/88286043 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f031c0051d0 con 0x7f032c106370 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 Processor -- start 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 -- start start 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x564900109a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900116bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56490006e970 con 0x564900109f40 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56490000b4a0 con 0x564900109620 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0336b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56490006a930 con 0x564900114800 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f032affd640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f03140a3dc0 0x7f03140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.970+0000 7f0331b12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57234/0 (socket says 172.21.15.37:57234) 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 -- 172.21.15.37:0/4215924285 learned_addr learned my addr 172.21.15.37:0/4215924285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331311640 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900116bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0332313640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f03140a3dc0 0x7f03140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331311640 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x564900109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f032affd640 1 -- 172.21.15.37:0/88286043 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f031c0d0950 con 0x7f032c106370 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 -- 172.21.15.37:0/4215924285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 msgr2=0x564900116bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900116bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 -- 172.21.15.37:0/4215924285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 msgr2=0x564900109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x564900109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 -- 172.21.15.37:0/4215924285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ffd81210 con 0x564900109f40 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0331b12640 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001142c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f032400cc20 tx=0x7f032400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2376ebf49b1ef9da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0332313640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f03140a3dc0 0x7f03140a6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f031c0ce110 tx=0x7f031c005890 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0313fff640 1 -- 172.21.15.37:0/4215924285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324014070 con 0x564900109f40 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0313fff640 1 -- 172.21.15.37:0/4215924285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03240047a0 con 0x564900109f40 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.971+0000 7f0313fff640 1 -- 172.21.15.37:0/4215924285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324010690 con 0x564900109f40 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 -- 172.21.15.37:0/4215924285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 msgr2=0x5649001142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001142c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f032400cc20 tx=0x7f032400e720 comp rx=0 tx=0).stop 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 -- 172.21.15.37:0/4215924285 shutdown_connections 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001142c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 --2- 172.21.15.37:0/4215924285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x564900109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 -- 172.21.15.37:0/4215924285 >> 172.21.15.37:0/4215924285 conn(0x564900105b30 msgr2=0x564900105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 -- 172.21.15.37:0/4215924285 shutdown_connections 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 -- 172.21.15.37:0/4215924285 wait complete. 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.972+0000 7f0336b69740 1 -- start start 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 Processor -- start 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x5649001140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900111900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564900117430 con 0x564900109f40 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5649000090b0 con 0x564900109620 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5649001175b0 con 0x564900114800 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0336b69740 10 client.0 _ll_register_callbacks cb 0x5648ffff6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001113c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57236/0 (socket says 172.21.15.37:57236) 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 -- 172.21.15.37:0/1347771347 learned_addr learned my addr 172.21.15.37:0/1347771347 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331311640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x5649001140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331311640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900111900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 -- 172.21.15.37:0/1347771347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 msgr2=0x564900111900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900111900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 -- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 msgr2=0x5649001140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x5649001140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331b12640 1 -- 172.21.15.37:0/1347771347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ffe4d620 con 0x564900109f40 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.973+0000 7f0331311640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0331b12640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001113c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0324005250 tx=0x7f0324005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f89b24c940c9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f03117fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f03117fa640 1 -- 172.21.15.37:0/1347771347 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324014070 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f03117fa640 1 -- 172.21.15.37:0/1347771347 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f0324010430 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03240047a0 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f03117fa640 1 -- 172.21.15.37:0/1347771347 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0324010670 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 10 client.16695 fetch_fsmap learned FSMap version 42 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5649001232d0 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f03117fa640 1 -- 172.21.15.37:0/1347771347 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03240047a0 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f03117fa640 1 -- 172.21.15.37:0/1347771347 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7f0324029030 con 0x564900109f40 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 10 client.16695 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 20 client.16695 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900124680 0x564900126a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 4 client.16695 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.974+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56490007c390 con 0x564900124680 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.975+0000 7f0331311640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900124680 0x564900126a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.975+0000 7f03117fa640 10 client.16695 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.975+0000 7f0331311640 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900124680 0x564900126a70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f2c5281b13d533 server_cookie=f71a46705fa7bd9f in_seq=0 out_seq=0 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.976+0000 7f03117fa640 1 -- 172.21.15.37:0/1347771347 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56490007c390 con 0x564900124680 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.976+0000 7f03117fa640 10 client.16695 handle_command_reply: tid=0 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.976+0000 7f0336b69740 1 client.16695 shutdown 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.976+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 msgr2=0x5649001113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.976+0000 7f0336b69740 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001113c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0324005250 tx=0x7f0324005280 comp rx=0 tx=0).stop 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 shutdown_connections 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900124680 0x564900126a70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900114800 0x564900111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f03117fa640 0 client.16695 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109f40 0x5649001113c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 --2- 172.21.15.37:0/1347771347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109620 0x5649001140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 >> 172.21.15.37:0/1347771347 conn(0x564900105b30 msgr2=0x56490010b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 shutdown_connections 2024-04-23T18:31:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 1 -- 172.21.15.37:0/1347771347 wait complete. 2024-04-23T18:31:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.977+0000 7f0336b69740 20 client.16695 trim_cache size 0 max 16384 2024-04-23T18:31:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.991+0000 7f0336b69740 1 Processor -- start 2024-04-23T18:31:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.991+0000 7f0336b69740 1 -- start start 2024-04-23T18:31:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0336b69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900105b30 0x5649001202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0336b69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 0x56490010c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0336b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109f40 0x564900113c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0336b69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b6b80 con 0x564900109620 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0336b69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b43f0 con 0x564900105b30 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0336b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b2260 con 0x564900109f40 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 0x56490010c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 0x56490010c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57242/0 (socket says 172.21.15.37:57242) 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 -- 172.21.15.37:0/2604810787 learned_addr learned my addr 172.21.15.37:0/2604810787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331311640 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900105b30 0x5649001202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 -- 172.21.15.37:0/2604810787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109f40 msgr2=0x564900113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109f40 0x564900113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 -- 172.21.15.37:0/2604810787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900105b30 msgr2=0x5649001202f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900105b30 0x5649001202f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 -- 172.21.15.37:0/2604810787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564900106b30 con 0x564900109620 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f0331b12640 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 0x56490010c850 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0324005d80 tx=0x7f032400c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bee4f09a57a116 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f03137fe640 1 -- 172.21.15.37:0/2604810787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324014070 con 0x564900109620 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f03137fe640 1 -- 172.21.15.37:0/2604810787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0324010430 con 0x564900109620 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.992+0000 7f03137fe640 1 -- 172.21.15.37:0/2604810787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324004040 con 0x564900109620 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 -- 172.21.15.37:0/2604810787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 msgr2=0x56490010c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 0x56490010c850 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0324005d80 tx=0x7f032400c530 comp rx=0 tx=0).stop 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 -- 172.21.15.37:0/2604810787 shutdown_connections 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109f40 0x564900113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900109620 0x56490010c850 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 --2- 172.21.15.37:0/2604810787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900105b30 0x5649001202f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 -- 172.21.15.37:0/2604810787 >> 172.21.15.37:0/2604810787 conn(0x564900111a60 msgr2=0x56490010c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 -- 172.21.15.37:0/2604810787 shutdown_connections 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 -- 172.21.15.37:0/2604810787 wait complete. 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 -- start start 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.993+0000 7f0336b69740 1 Processor -- start 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 0x56490011c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109620 0x56490011cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109f40 0x56490010d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b28b0 con 0x564900105b30 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b3d20 con 0x564900109f40 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1b2f80 con 0x564900109620 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0336b69740 10 client.0 _ll_register_callbacks cb 0x7f0324004da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 0x56490011c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 0x56490011c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57258/0 (socket says 172.21.15.37:57258) 2024-04-23T18:31:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 -- 172.21.15.37:0/1506281769 learned_addr learned my addr 172.21.15.37:0/1506281769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331b12640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109620 0x56490011cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109f40 0x56490010d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 -- 172.21.15.37:0/1506281769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109620 msgr2=0x56490011cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109620 0x56490011cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 -- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109f40 msgr2=0x56490010d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109f40 0x56490010d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 -- 172.21.15.37:0/1506281769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56490010dea0 con 0x564900105b30 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f0331311640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 0x56490011c800 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0320000c00 tx=0x7f0320009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a0090b3ce64337 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.994+0000 7f03117fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f03117fa640 1 -- 172.21.15.37:0/1506281769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320035040 con 0x564900105b30 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f03117fa640 1 -- 172.21.15.37:0/1506281769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f0320005020 con 0x564900105b30 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0324010430 con 0x564900105b30 2024-04-23T18:31:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f03117fa640 1 -- 172.21.15.37:0/1506281769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0320005220 con 0x564900105b30 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 10 client.16701 fetch_fsmap learned FSMap version 42 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56490018a560 con 0x564900105b30 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f03117fa640 1 -- 172.21.15.37:0/1506281769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0324010430 con 0x564900105b30 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f03117fa640 1 -- 172.21.15.37:0/1506281769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7f032003a040 con 0x564900105b30 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 10 client.16701 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 20 client.16701 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900114800 0x56490018cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 4 client.16701 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564900076200 con 0x564900114800 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.995+0000 7f0331b12640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900114800 0x56490018cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.996+0000 7f0331b12640 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900114800 0x56490018cd20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f44473cfa69dcb server_cookie=1bf211ac7304bfae in_seq=0 out_seq=0 2024-04-23T18:31:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:00.996+0000 7f03117fa640 10 client.16701 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.034+0000 7f03117fa640 1 -- 172.21.15.37:0/1506281769 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564900076200 con 0x564900114800 2024-04-23T18:31:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.034+0000 7f03117fa640 10 client.16701 handle_command_reply: tid=0 2024-04-23T18:31:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.034+0000 7f0336b69740 1 client.16701 shutdown 2024-04-23T18:31:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.034+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 msgr2=0x56490011c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.034+0000 7f0336b69740 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 0x56490011c800 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0320000c00 tx=0x7f0320009260 comp rx=0 tx=0).stop 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 shutdown_connections 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564900114800 0x56490018cd20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564900109f40 0x56490010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564900109620 0x56490011cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f03117fa640 0 client.16701 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 --2- 172.21.15.37:0/1506281769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564900105b30 0x56490011c800 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 >> 172.21.15.37:0/1506281769 conn(0x564900111a60 msgr2=0x56490010c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 shutdown_connections 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 1 -- 172.21.15.37:0/1506281769 wait complete. 2024-04-23T18:31:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0336b69740 20 client.16701 trim_cache size 0 max 16384 2024-04-23T18:31:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:31:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:31:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:31:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f03140a3dc0 msgr2=0x7f03140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0333d76640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f03140a3dc0 0x7f03140a6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f031c0ce110 tx=0x7f031c005890 comp rx=0 tx=0).stop 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 msgr2=0x7f032c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.035+0000 7f0333d76640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 0x7f032c19eac0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f031c00cad0 tx=0x7f031c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 shutdown_connections 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f03140a3dc0 0x7f03140a6280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f032c106370 0x7f032c19eac0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f032c101890 0x7f032c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 --2- 172.21.15.37:0/88286043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f032c100f70 0x7f032c1a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 >> 172.21.15.37:0/88286043 conn(0x7f032c077fd0 msgr2=0x7f032c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 shutdown_connections 2024-04-23T18:31:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:01.036+0000 7f0333d76640 1 -- 172.21.15.37:0/88286043 wait complete. 2024-04-23T18:31:01.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 518324 3.57 MB/sec execute 426 sec latency 575.514 ms 2024-04-23T18:31:02.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:02 smithi172 ceph-mon[32456]: pgmap v329: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-23T18:31:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:02 smithi173 ceph-mon[32906]: pgmap v329: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-23T18:31:02.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:02 smithi037 ceph-mon[29404]: pgmap v329: 97 pgs: 97 active+clean; 356 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-04-23T18:31:02.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 519651 3.57 MB/sec execute 427 sec latency 22.458 ms 2024-04-23T18:31:03.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 521103 3.58 MB/sec execute 428 sec latency 365.401 ms 2024-04-23T18:31:04.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:04 smithi172 ceph-mon[32456]: pgmap v330: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 216 op/s 2024-04-23T18:31:04.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:04 smithi173 ceph-mon[32906]: pgmap v330: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 216 op/s 2024-04-23T18:31:04.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:04 smithi037 ceph-mon[29404]: pgmap v330: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 216 op/s 2024-04-23T18:31:04.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 522686 3.60 MB/sec execute 429 sec latency 15.392 ms 2024-04-23T18:31:05.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:05 smithi172 ceph-mon[32456]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:31:05.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:05 smithi173 ceph-mon[32906]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:31:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:05 smithi037 ceph-mon[29404]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:31:05.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 523906 3.60 MB/sec execute 430 sec latency 249.524 ms 2024-04-23T18:31:06.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:06 smithi172 ceph-mon[32456]: pgmap v332: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-04-23T18:31:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:06 smithi173 ceph-mon[32906]: pgmap v332: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-04-23T18:31:06.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:06 smithi037 ceph-mon[29404]: pgmap v332: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-04-23T18:31:06.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 523906 3.59 MB/sec execute 431 sec latency 1249.626 ms 2024-04-23T18:31:07.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 523906 3.59 MB/sec execute 432 sec latency 2249.712 ms 2024-04-23T18:31:08.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:08 smithi172 ceph-mon[32456]: pgmap v333: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:08.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:08 smithi173 ceph-mon[32906]: pgmap v333: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:08.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:08 smithi037 ceph-mon[29404]: pgmap v333: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:08.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 524622 3.58 MB/sec execute 433 sec latency 2954.882 ms 2024-04-23T18:31:09.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 526310 3.60 MB/sec execute 434 sec latency 13.405 ms 2024-04-23T18:31:10.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:10 smithi172 ceph-mon[32456]: pgmap v334: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:10.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:10 smithi173 ceph-mon[32906]: pgmap v334: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:10.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:10 smithi037 ceph-mon[29404]: pgmap v334: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:10.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 527456 3.60 MB/sec execute 435 sec latency 325.367 ms 2024-04-23T18:31:11.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 527456 3.60 MB/sec execute 436 sec latency 1325.504 ms 2024-04-23T18:31:12.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:12 smithi172 ceph-mon[32456]: pgmap v335: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:12 smithi172 ceph-mon[32456]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:31:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:12 smithi173 ceph-mon[32906]: pgmap v335: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:12 smithi173 ceph-mon[32906]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:31:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:12 smithi037 ceph-mon[29404]: pgmap v335: 97 pgs: 97 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-04-23T18:31:12.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:12 smithi037 ceph-mon[29404]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:31:12.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 528205 3.59 MB/sec execute 437 sec latency 2053.043 ms 2024-04-23T18:31:13.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 529900 3.61 MB/sec execute 438 sec latency 24.870 ms 2024-04-23T18:31:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 -- 172.21.15.37:0/4210417411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c1031e0 msgr2=0x7fc10c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 --2- 172.21.15.37:0/4210417411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c1031e0 0x7fc10c1035c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc108009ca0 tx=0x7fc10802a9e0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 -- 172.21.15.37:0/4210417411 shutdown_connections 2024-04-23T18:31:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 --2- 172.21.15.37:0/4210417411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc10c10c520 0x7fc10c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 --2- 172.21.15.37:0/4210417411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c103b00 0x7fc10c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 --2- 172.21.15.37:0/4210417411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c1031e0 0x7fc10c1035c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 -- 172.21.15.37:0/4210417411 >> 172.21.15.37:0/4210417411 conn(0x7fc10c077fd0 msgr2=0x7fc10c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 -- 172.21.15.37:0/4210417411 shutdown_connections 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.136+0000 7fc11455d640 1 -- 172.21.15.37:0/4210417411 wait complete. 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 Processor -- start 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 -- start start 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc10c1031e0 0x7fc10c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 0x7fc10c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c10c520 0x7fc10c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c112190 con 0x7fc10c103b00 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c112010 con 0x7fc10c1031e0 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc11455d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c112310 con 0x7fc10c10c520 2024-04-23T18:31:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 0x7fc10c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 0x7fc10c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33016/0 (socket says 172.21.15.37:33016) 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 -- 172.21.15.37:0/696960420 learned_addr learned my addr 172.21.15.37:0/696960420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc112afa640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c10c520 0x7fc10c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc1122f9640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc10c1031e0 0x7fc10c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 -- 172.21.15.37:0/696960420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c10c520 msgr2=0x7fc10c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c10c520 0x7fc10c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 -- 172.21.15.37:0/696960420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc10c1031e0 msgr2=0x7fc10c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.137+0000 7fc111af8640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc10c1031e0 0x7fc10c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc111af8640 1 -- 172.21.15.37:0/696960420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c19d6a0 con 0x7fc10c103b00 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc111af8640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 0x7fc10c198bf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00ea60 tx=0x7fc0fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc1037fe640 1 -- 172.21.15.37:0/696960420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00cde0 con 0x7fc10c103b00 2024-04-23T18:31:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c19d990 con 0x7fc10c103b00 2024-04-23T18:31:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c1a5220 con 0x7fc10c103b00 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc1037fe640 1 -- 172.21.15.37:0/696960420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc0035f0 con 0x7fc10c103b00 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.138+0000 7fc1037fe640 1 -- 172.21.15.37:0/696960420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc018690 con 0x7fc10c103b00 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc1037fe640 1 -- 172.21.15.37:0/696960420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc0fc004020 con 0x7fc10c103b00 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 Processor -- start 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 -- start start 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc514a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc521bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2dc479970 con 0x55d2dc514f40 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.141+0000 7fc11727c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2dc4164a0 con 0x55d2dc514620 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc11727c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d2dc475930 con 0x55d2dc51f800 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33032/0 (socket says 172.21.15.37:33032) 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 -- 172.21.15.37:0/3296980561 learned_addr learned my addr 172.21.15.37:0/3296980561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc111af8640 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc521bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc112afa640 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc514a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 -- 172.21.15.37:0/3296980561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 msgr2=0x55d2dc521bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc521bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1037fe640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0f80a3dc0 0x7fc0f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 -- 172.21.15.37:0/3296980561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 msgr2=0x55d2dc514a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc514a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 -- 172.21.15.37:0/3296980561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dc18c210 con 0x55d2dc514f40 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc111af8640 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc521bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc112afa640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0f80a3dc0 0x7fc0f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1122f9640 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51f2c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc108009410 tx=0x7fc108005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeebaa43a86a008 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc1037fe640 1 -- 172.21.15.37:0/696960420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fc0fc0d0990 con 0x7fc10c103b00 2024-04-23T18:31:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.142+0000 7fc100bfe640 1 -- 172.21.15.37:0/3296980561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108049070 con 0x55d2dc514f40 2024-04-23T18:31:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc112afa640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0f80a3dc0 0x7fc0f80a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b6e0 tx=0x7fc1040046c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc100bfe640 1 -- 172.21.15.37:0/3296980561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10802ee80 con 0x55d2dc514f40 2024-04-23T18:31:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc100bfe640 1 -- 172.21.15.37:0/3296980561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108033040 con 0x55d2dc514f40 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 -- 172.21.15.37:0/3296980561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 msgr2=0x55d2dc51f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51f2c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc108009410 tx=0x7fc108005ef0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 -- 172.21.15.37:0/3296980561 shutdown_connections 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc521bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51f2c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 --2- 172.21.15.37:0/3296980561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc514a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.143+0000 7fc11727c740 1 -- 172.21.15.37:0/3296980561 >> 172.21.15.37:0/3296980561 conn(0x55d2dc510b30 msgr2=0x55d2dc510f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 -- 172.21.15.37:0/3296980561 shutdown_connections 2024-04-23T18:31:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 -- 172.21.15.37:0/3296980561 wait complete. 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 -- start start 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 Processor -- start 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc51f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc51f800 0x55d2dc51c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2dc5223d0 con 0x55d2dc514f40 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2dc4140b0 con 0x55d2dc51f800 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d2dc522550 con 0x55d2dc514620 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc11727c740 10 client.0 _ll_register_callbacks cb 0x55d2dc401d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc111af8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc1122f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc51f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc111af8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33042/0 (socket says 172.21.15.37:33042) 2024-04-23T18:31:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc111af8640 1 -- 172.21.15.37:0/1666295164 learned_addr learned my addr 172.21.15.37:0/1666295164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.144+0000 7fc112afa640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc51f800 0x55d2dc51c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc1122f9640 1 -- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc51f800 msgr2=0x55d2dc51c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc1122f9640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc51f800 0x55d2dc51c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc1122f9640 1 -- 172.21.15.37:0/1666295164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 msgr2=0x55d2dc51c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc1122f9640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc1122f9640 1 -- 172.21.15.37:0/1666295164 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dc258620 con 0x55d2dc514620 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc111af8640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc1122f9640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc51f0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc108006010 tx=0x7fc10802b0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947684d7103ecefa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/1666295164 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108051020 con 0x55d2dc514620 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/1666295164 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fc108003970 con 0x55d2dc514620 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10802ee80 con 0x55d2dc514620 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/1666295164 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc108059050 con 0x55d2dc514620 2024-04-23T18:31:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc11727c740 10 client.26137 fetch_fsmap learned FSMap version 42 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2dc5223d0 con 0x55d2dc514620 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.145+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/1666295164 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10802ee80 con 0x55d2dc514620 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/1666295164 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fc108043070 con 0x55d2dc514620 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc11727c740 10 client.26137 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc11727c740 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc11727c740 20 client.26137 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc11727c740 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc52f420 0x55d2dc531810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc11727c740 4 client.26137 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2dc487390 con 0x55d2dc52f420 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc111af8640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc52f420 0x55d2dc531810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc111af8640 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc52f420 0x55d2dc531810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe6392d47d1a1af server_cookie=2bd9deb7a9f2f7d2 in_seq=0 out_seq=0 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.146+0000 7fc0f5ffb640 10 client.26137 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/1666295164 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2dc487390 con 0x55d2dc52f420 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc0f5ffb640 10 client.26137 handle_command_reply: tid=0 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 client.26137 shutdown 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 msgr2=0x55d2dc51f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc51f0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc108006010 tx=0x7fc10802b0e0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 shutdown_connections 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc52f420 0x55d2dc531810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc51f800 0x55d2dc51c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc51c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc0f5ffb640 0 client.26137 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 --2- 172.21.15.37:0/1666295164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc51f0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.147+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 >> 172.21.15.37:0/1666295164 conn(0x55d2dc510b30 msgr2=0x55d2dc5168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.148+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 shutdown_connections 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.148+0000 7fc11727c740 1 -- 172.21.15.37:0/1666295164 wait complete. 2024-04-23T18:31:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.148+0000 7fc11727c740 20 client.26137 trim_cache size 0 max 16384 2024-04-23T18:31:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.163+0000 7fc11727c740 1 Processor -- start 2024-04-23T18:31:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.163+0000 7fc11727c740 1 -- start start 2024-04-23T18:31:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.163+0000 7fc11727c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc52b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc11727c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514f40 0x55d2dc51ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc11727c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 0x55d2dc537030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc11727c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0011180 con 0x55d2dc51f800 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc11727c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f000e9f0 con 0x55d2dc514f40 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc11727c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f000c860 con 0x55d2dc514620 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 0x55d2dc537030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc1122f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc52b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 0x55d2dc537030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33050/0 (socket says 172.21.15.37:33050) 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 -- 172.21.15.37:0/3099627525 learned_addr learned my addr 172.21.15.37:0/3099627525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc111af8640 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514f40 0x55d2dc51ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 -- 172.21.15.37:0/3099627525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 msgr2=0x55d2dc52b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc52b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 -- 172.21.15.37:0/3099627525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514f40 msgr2=0x55d2dc51ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514f40 0x55d2dc51ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 -- 172.21.15.37:0/3099627525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dc514d70 con 0x55d2dc51f800 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc1122f9640 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc52b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc112afa640 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 0x55d2dc537030 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc104003e60 tx=0x7fc104016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c547c5b840efaa09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc0f7fff640 1 -- 172.21.15.37:0/3099627525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104003f50 con 0x55d2dc51f800 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc0f7fff640 1 -- 172.21.15.37:0/3099627525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc104011d10 con 0x55d2dc51f800 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.164+0000 7fc0f7fff640 1 -- 172.21.15.37:0/3099627525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10401f780 con 0x55d2dc51f800 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 -- 172.21.15.37:0/3099627525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 msgr2=0x55d2dc537030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 0x55d2dc537030 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc104003e60 tx=0x7fc104016000 comp rx=0 tx=0).stop 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 -- 172.21.15.37:0/3099627525 shutdown_connections 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc51f800 0x55d2dc537030 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514f40 0x55d2dc51ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 --2- 172.21.15.37:0/3099627525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc514620 0x55d2dc52b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 -- 172.21.15.37:0/3099627525 >> 172.21.15.37:0/3099627525 conn(0x55d2dc510b30 msgr2=0x55d2dc51c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 -- 172.21.15.37:0/3099627525 shutdown_connections 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.165+0000 7fc11727c740 1 -- 172.21.15.37:0/3099627525 wait complete. 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 -- start start 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 Processor -- start 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc5232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc523800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc523d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f000ceb0 con 0x55d2dc514f40 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f000e320 con 0x55d2dc514620 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f000d580 con 0x55d2dc51f800 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc11727c740 10 client.0 _ll_register_callbacks cb 0x7fc0fc0d8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc111af8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc523800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc111af8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc523800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33052/0 (socket says 172.21.15.37:33052) 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc112afa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc523d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc111af8640 1 -- 172.21.15.37:0/2534779434 learned_addr learned my addr 172.21.15.37:0/2534779434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.166+0000 7fc1122f9640 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc5232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc111af8640 1 -- 172.21.15.37:0/2534779434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 msgr2=0x55d2dc523d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc111af8640 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc523d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc111af8640 1 -- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 msgr2=0x55d2dc5232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc111af8640 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc5232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc111af8640 1 -- 172.21.15.37:0/2534779434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2dc594950 con 0x55d2dc514f40 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc111af8640 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc523800 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0992c0 tx=0x7fc0fc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b18d2cd290c85c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/2534779434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc09c250 con 0x55d2dc514f40 2024-04-23T18:31:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/2534779434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fc0fc004310 con 0x55d2dc514f40 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc104011d10 con 0x55d2dc514f40 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/2534779434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc09cff0 con 0x55d2dc514f40 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc11727c740 10 client.16719 fetch_fsmap learned FSMap version 42 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2dc595ca0 con 0x55d2dc514f40 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.167+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/2534779434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc104011d10 con 0x55d2dc514f40 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/2534779434 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7fc0fc0a50d0 con 0x55d2dc514f40 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc11727c740 10 client.16719 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc11727c740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc11727c740 20 client.16719 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc11727c740 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc596430 0x55d2dc598850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc11727c740 4 client.16719 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2dc481200 con 0x55d2dc596430 2024-04-23T18:31:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc1122f9640 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc596430 0x55d2dc598850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc0f5ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.168+0000 7fc1122f9640 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc596430 0x55d2dc598850 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5abc0fca78a784 server_cookie=9457941db21e8e61 in_seq=0 out_seq=0 2024-04-23T18:31:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc0f5ffb640 1 -- 172.21.15.37:0/2534779434 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2dc481200 con 0x55d2dc596430 2024-04-23T18:31:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc0f5ffb640 10 client.16719 handle_command_reply: tid=0 2024-04-23T18:31:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 client.16719 shutdown 2024-04-23T18:31:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 msgr2=0x55d2dc523800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc523800 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0992c0 tx=0x7fc0fc0095b0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 shutdown_connections 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d2dc596430 0x55d2dc598850 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2dc51f800 0x55d2dc523d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc0f5ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2dc514f40 0x55d2dc523800 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 --2- 172.21.15.37:0/2534779434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2dc514620 0x55d2dc5232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 >> 172.21.15.37:0/2534779434 conn(0x55d2dc510b30 msgr2=0x55d2dc515f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 shutdown_connections 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 1 -- 172.21.15.37:0/2534779434 wait complete. 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.212+0000 7fc11727c740 20 client.16719 trim_cache size 0 max 16384 2024-04-23T18:31:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:31:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:31:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:31:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:31:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0f80a3dc0 msgr2=0x7fc0f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0f80a3dc0 0x7fc0f80a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b6e0 tx=0x7fc1040046c0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 msgr2=0x7fc10c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 0x7fc10c198bf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00ea60 tx=0x7fc0fc00ef20 comp rx=0 tx=0).stop 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 shutdown_connections 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0f80a3dc0 0x7fc0f80a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c10c520 0x7fc10c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc10c103b00 0x7fc10c198bf0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 --2- 172.21.15.37:0/696960420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc10c1031e0 0x7fc10c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 >> 172.21.15.37:0/696960420 conn(0x7fc10c077fd0 msgr2=0x7fc10c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 shutdown_connections 2024-04-23T18:31:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:14.213+0000 7fc11455d640 1 -- 172.21.15.37:0/696960420 wait complete. 2024-04-23T18:31:14.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:14 smithi172 ceph-mon[32456]: pgmap v337: 97 pgs: 97 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2024-04-23T18:31:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:14 smithi173 ceph-mon[32906]: pgmap v337: 97 pgs: 97 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2024-04-23T18:31:14.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:14 smithi037 ceph-mon[29404]: pgmap v337: 97 pgs: 97 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2024-04-23T18:31:14.847 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530999 3.62 MB/sec execute 439 sec latency 320.070 ms 2024-04-23T18:31:15.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530999 3.61 MB/sec execute 440 sec latency 1320.195 ms 2024-04-23T18:31:16.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:16 smithi172 ceph-mon[32456]: pgmap v338: 97 pgs: 97 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-23T18:31:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:16 smithi173 ceph-mon[32906]: pgmap v338: 97 pgs: 97 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-23T18:31:16.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:16 smithi037 ceph-mon[29404]: pgmap v338: 97 pgs: 97 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-23T18:31:16.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530999 3.60 MB/sec execute 441 sec latency 2320.268 ms 2024-04-23T18:31:17.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 530999 3.59 MB/sec execute 442 sec latency 3320.359 ms 2024-04-23T18:31:18.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:18 smithi172 ceph-mon[32456]: pgmap v339: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:18 smithi173 ceph-mon[32906]: pgmap v339: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:18.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:18 smithi037 ceph-mon[29404]: pgmap v339: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:18.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 531782 3.59 MB/sec execute 443 sec latency 4026.985 ms 2024-04-23T18:31:19.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 533439 3.61 MB/sec execute 444 sec latency 15.761 ms 2024-04-23T18:31:20.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:20 smithi172 ceph-mon[32456]: pgmap v340: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:20 smithi173 ceph-mon[32906]: pgmap v340: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:20.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:20 smithi037 ceph-mon[29404]: pgmap v340: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:20.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 534579 3.61 MB/sec execute 445 sec latency 321.975 ms 2024-04-23T18:31:21.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 534579 3.60 MB/sec execute 446 sec latency 1322.106 ms 2024-04-23T18:31:22.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:22 smithi172 ceph-mon[32456]: pgmap v341: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:22 smithi173 ceph-mon[32906]: pgmap v341: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:22.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:22 smithi037 ceph-mon[29404]: pgmap v341: 97 pgs: 97 active+clean; 372 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-04-23T18:31:22.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 535330 3.60 MB/sec execute 447 sec latency 2048.555 ms 2024-04-23T18:31:23.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 537107 3.62 MB/sec execute 448 sec latency 12.211 ms 2024-04-23T18:31:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:24 smithi172 ceph-mon[32456]: pgmap v342: 97 pgs: 97 active+clean; 377 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-04-23T18:31:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:24 smithi173 ceph-mon[32906]: pgmap v342: 97 pgs: 97 active+clean; 377 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-04-23T18:31:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:24 smithi037 ceph-mon[29404]: pgmap v342: 97 pgs: 97 active+clean; 377 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-04-23T18:31:24.848 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538122 3.62 MB/sec execute 449 sec latency 416.128 ms 2024-04-23T18:31:25.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538122 3.61 MB/sec execute 450 sec latency 1416.208 ms 2024-04-23T18:31:26.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:26 smithi172 ceph-mon[32456]: pgmap v343: 97 pgs: 97 active+clean; 377 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-04-23T18:31:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:26 smithi173 ceph-mon[32906]: pgmap v343: 97 pgs: 97 active+clean; 377 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-04-23T18:31:26.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:26 smithi037 ceph-mon[29404]: pgmap v343: 97 pgs: 97 active+clean; 377 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-04-23T18:31:26.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 538122 3.60 MB/sec execute 451 sec latency 2416.375 ms 2024-04-23T18:31:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/1851770570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 msgr2=0x7ff4d4112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/1851770570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d4112a20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009900 tx=0x7ff4c402a480 comp rx=0 tx=0).stop 2024-04-23T18:31:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/1851770570 shutdown_connections 2024-04-23T18:31:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/1851770570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4d4112f60 0x7ff4d4115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/1851770570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d4112a20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/1851770570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d40742f0 0x7ff4d40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/1851770570 >> 172.21.15.37:0/1851770570 conn(0x7ff4d40fda20 msgr2=0x7ff4d40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/1851770570 shutdown_connections 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/1851770570 wait complete. 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 Processor -- start 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.311+0000 7ff4dbdd8640 1 -- start start 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4dbdd8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4d40742f0 0x7ff4d41a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4dbdd8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d41a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4dbdd8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4112f60 0x7ff4d419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4dbdd8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41187f0 con 0x7ff4d4074c10 2024-04-23T18:31:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4dbdd8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4118670 con 0x7ff4d40742f0 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4dbdd8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4118970 con 0x7ff4d4112f60 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d41a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d41a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40830/0 (socket says 172.21.15.37:40830) 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 -- 172.21.15.37:0/366548115 learned_addr learned my addr 172.21.15.37:0/366548115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4d40742f0 0x7ff4d41a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4da375640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4112f60 0x7ff4d419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 -- 172.21.15.37:0/366548115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4112f60 msgr2=0x7ff4d419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4112f60 0x7ff4d419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 -- 172.21.15.37:0/366548115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4d40742f0 msgr2=0x7ff4d41a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4d40742f0 0x7ff4d41a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 -- 172.21.15.37:0/366548115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d419f3f0 con 0x7ff4d4074c10 2024-04-23T18:31:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.312+0000 7ff4d9373640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d41a4fb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005940 tx=0x7ff4c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.313+0000 7ff4caffd640 1 -- 172.21.15.37:0/366548115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c403f030 con 0x7ff4d4074c10 2024-04-23T18:31:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.313+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d419f680 con 0x7ff4d4074c10 2024-04-23T18:31:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.313+0000 7ff4caffd640 1 -- 172.21.15.37:0/366548115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c4003aa0 con 0x7ff4d4074c10 2024-04-23T18:31:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.313+0000 7ff4caffd640 1 -- 172.21.15.37:0/366548115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4035070 con 0x7ff4d4074c10 2024-04-23T18:31:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.313+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d41ab810 con 0x7ff4d4074c10 2024-04-23T18:31:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.315+0000 7ff4caffd640 1 -- 172.21.15.37:0/366548115 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4c4042030 con 0x7ff4d4074c10 2024-04-23T18:31:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 Processor -- start 2024-04-23T18:31:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 -- start start 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b925864a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b92586f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925871bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b9257c9970 con 0x55b925864f40 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9257664a0 con 0x55b92586f800 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4debc7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b9257c5930 con 0x55b925864620 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4d9b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b92586f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4caffd640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4d9b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b92586f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40840/0 (socket says 172.21.15.37:40840) 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.316+0000 7ff4d9b74640 1 -- 172.21.15.37:0/2175508365 learned_addr learned my addr 172.21.15.37:0/2175508365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4da375640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9373640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925871bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4da375640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b925864a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9b74640 1 -- 172.21.15.37:0/2175508365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 msgr2=0x55b925864a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b925864a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9b74640 1 -- 172.21.15.37:0/2175508365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 msgr2=0x55b925871bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925871bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4caffd640 1 -- 172.21.15.37:0/366548115 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7ff4c40f0ed0 con 0x7ff4d4074c10 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9b74640 1 -- 172.21.15.37:0/2175508365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9254dc210 con 0x55b925864f40 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9373640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925871bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4da375640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b925864a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b92586f2c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009d20 tx=0x7ff4d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbcabb39069ee45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4bffff640 1 -- 172.21.15.37:0/2175508365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001e070 con 0x55b925864f40 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4da375640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009370 tx=0x7ff4cc004a40 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4bffff640 1 -- 172.21.15.37:0/2175508365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0019ae0 con 0x55b925864f40 2024-04-23T18:31:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.317+0000 7ff4bffff640 1 -- 172.21.15.37:0/2175508365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000eac0 con 0x55b925864f40 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 -- 172.21.15.37:0/2175508365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 msgr2=0x55b92586f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b92586f2c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009d20 tx=0x7ff4d0011810 comp rx=0 tx=0).stop 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 -- 172.21.15.37:0/2175508365 shutdown_connections 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925871bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b92586f2c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 --2- 172.21.15.37:0/2175508365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b925864a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 -- 172.21.15.37:0/2175508365 >> 172.21.15.37:0/2175508365 conn(0x55b925860b30 msgr2=0x55b925860f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 -- 172.21.15.37:0/2175508365 shutdown_connections 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 -- 172.21.15.37:0/2175508365 wait complete. 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 -- start start 2024-04-23T18:31:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.318+0000 7ff4debc7740 1 Processor -- start 2024-04-23T18:31:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b92586c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b92586c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b925872430 con 0x55b925864620 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9257640b0 con 0x55b92586f800 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b9258725b0 con 0x55b925864f40 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4debc7740 10 client.0 _ll_register_callbacks cb 0x55b925751d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40844/0 (socket says 172.21.15.37:40844) 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1684867766 learned_addr learned my addr 172.21.15.37:0/1684867766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9373640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b92586c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4da375640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b92586c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1684867766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 msgr2=0x55b92586c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b92586c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 msgr2=0x55b92586c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b92586c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.319+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1684867766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9255a8620 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586f060 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e760 tx=0x7ff4d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a114f1fa47efe1d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/1684867766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0027020 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/1684867766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7ff4d0019630 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d0019ae0 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/1684867766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff4d0032050 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 10 client.16731 fetch_fsmap learned FSMap version 42 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b92587e1c0 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/1684867766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0019ae0 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/1684867766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7ff4d000ec10 con 0x55b925864620 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 10 client.16731 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 20 client.16731 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b92587f5c0 0x55b9258819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 4 client.16731 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.320+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9257d7390 con 0x55b92587f5c0 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.321+0000 7ff4d9373640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b92587f5c0 0x55b9258819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.321+0000 7ff4d9373640 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b92587f5c0 0x55b9258819b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171f3e842c06392c server_cookie=3eced0667c6d74da in_seq=0 out_seq=0 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.321+0000 7ff4bd7fa640 10 client.16731 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/1684867766 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9257d7390 con 0x55b92587f5c0 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4bd7fa640 10 client.16731 handle_command_reply: tid=0 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 client.16731 shutdown 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 msgr2=0x55b92586f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586f060 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e760 tx=0x7ff4d0004270 comp rx=0 tx=0).stop 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 shutdown_connections 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b92587f5c0 0x55b9258819b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b92586c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b92586c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4bd7fa640 0 client.16731 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1684867766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586f060 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 >> 172.21.15.37:0/1684867766 conn(0x55b925860b30 msgr2=0x55b925866860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 shutdown_connections 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 1 -- 172.21.15.37:0/1684867766 wait complete. 2024-04-23T18:31:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.322+0000 7ff4debc7740 20 client.16731 trim_cache size 0 max 16384 2024-04-23T18:31:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.337+0000 7ff4debc7740 1 Processor -- start 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.337+0000 7ff4debc7740 1 -- start start 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b9258870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925887630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4debc7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41b6b10 con 0x55b925864620 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4debc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41b4380 con 0x55b92586f800 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4debc7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41b21f0 con 0x55b925864f40 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40858/0 (socket says 172.21.15.37:40858) 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1970616645 learned_addr learned my addr 172.21.15.37:0/1970616645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1970616645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 msgr2=0x55b9258870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b9258870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1970616645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 msgr2=0x55b925887630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925887630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 -- 172.21.15.37:0/1970616645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b925864d70 con 0x55b925864620 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586eb90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0011850 tx=0x7ff4d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e64d122602b897d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4bf7fe640 1 -- 172.21.15.37:0/1970616645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0027070 con 0x55b925864620 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4bf7fe640 1 -- 172.21.15.37:0/1970616645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0015020 con 0x55b925864620 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.338+0000 7ff4bf7fe640 1 -- 172.21.15.37:0/1970616645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001e070 con 0x55b925864620 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 -- 172.21.15.37:0/1970616645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 msgr2=0x55b92586eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586eb90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0011850 tx=0x7ff4d0005f90 comp rx=0 tx=0).stop 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 -- 172.21.15.37:0/1970616645 shutdown_connections 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b925887630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864f40 0x55b9258870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 --2- 172.21.15.37:0/1970616645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864620 0x55b92586eb90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 -- 172.21.15.37:0/1970616645 >> 172.21.15.37:0/1970616645 conn(0x55b925860b30 msgr2=0x55b92585fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 -- 172.21.15.37:0/1970616645 shutdown_connections 2024-04-23T18:31:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 -- 172.21.15.37:0/1970616645 wait complete. 2024-04-23T18:31:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 -- start start 2024-04-23T18:31:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.339+0000 7ff4debc7740 1 Processor -- start 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b9258778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b925874c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b9258751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41b2840 con 0x55b925864f40 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41b3cb0 con 0x55b92586f800 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41b2f10 con 0x55b925864620 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4debc7740 10 client.0 _ll_register_callbacks cb 0x7ff4d0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b925874c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b925874c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40874/0 (socket says 172.21.15.37:40874) 2024-04-23T18:31:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 -- 172.21.15.37:0/3128689911 learned_addr learned my addr 172.21.15.37:0/3128689911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b9258778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4da375640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b9258751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 -- 172.21.15.37:0/3128689911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 msgr2=0x55b9258778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b9258778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 -- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 msgr2=0x55b9258751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b9258751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 -- 172.21.15.37:0/3128689911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b925875700 con 0x55b925864f40 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4da375640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b9258751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4d9373640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b925874c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4006010 tx=0x7ff4c4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d42c8a0e234a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.340+0000 7ff4bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/3128689911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4049040 con 0x55b925864f40 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/3128689911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7ff4c40b94f0 con 0x55b925864f40 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d0015020 con 0x55b925864f40 2024-04-23T18:31:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/3128689911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff4c40c1180 con 0x55b925864f40 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 10 client.16737 fetch_fsmap learned FSMap version 42 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b9258e5710 con 0x55b925864f40 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/3128689911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0015020 con 0x55b925864f40 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/3128689911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 42) ==== 8354+0+0 (secure 0 0 0) 0x7ff4c40ee800 con 0x55b925864f40 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 10 client.16737 fetch_fsmap finished waiting for FSMap version 42 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 20 client.16737 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b9258e5ef0 0x55b9258e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 4 client.16737 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.341+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9257d1200 con 0x55b9258e5ef0 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.342+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b9258e5ef0 0x55b9258e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.342+0000 7ff4bd7fa640 10 client.16737 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.342+0000 7ff4d9b74640 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b9258e5ef0 0x55b9258e8310 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1fc31578029d425 server_cookie=eac5ee80a6467599 in_seq=0 out_seq=0 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.377+0000 7ff4bd7fa640 1 -- 172.21.15.37:0/3128689911 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9257d1200 con 0x55b9258e5ef0 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.377+0000 7ff4bd7fa640 10 client.16737 handle_command_reply: tid=0 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.377+0000 7ff4debc7740 1 client.16737 shutdown 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.377+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 msgr2=0x55b925874c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.377+0000 7ff4debc7740 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b925874c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4006010 tx=0x7ff4c4047000 comp rx=0 tx=0).stop 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 shutdown_connections 2024-04-23T18:31:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b9258e5ef0 0x55b9258e8310 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b92586f800 0x55b9258751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b925864f40 0x55b925874c80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 --2- 172.21.15.37:0/3128689911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b925864620 0x55b9258778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4bd7fa640 0 client.16737 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 >> 172.21.15.37:0/3128689911 conn(0x55b925860b30 msgr2=0x55b92585f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 shutdown_connections 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 1 -- 172.21.15.37:0/3128689911 wait complete. 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4debc7740 20 client.16737 trim_cache size 0 max 16384 2024-04-23T18:31:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:31:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:31:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:31:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:31:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff4c00a3dc0 msgr2=0x7ff4c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009370 tx=0x7ff4cc004a40 comp rx=0 tx=0).stop 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 msgr2=0x7ff4d41a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d41a4fb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005940 tx=0x7ff4c4004270 comp rx=0 tx=0).stop 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 shutdown_connections 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4112f60 0x7ff4d419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d4074c10 0x7ff4d41a4fb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 --2- 172.21.15.37:0/366548115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4d40742f0 0x7ff4d41a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 >> 172.21.15.37:0/366548115 conn(0x7ff4d40fda20 msgr2=0x7ff4d40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 shutdown_connections 2024-04-23T18:31:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:27.378+0000 7ff4dbdd8640 1 -- 172.21.15.37:0/366548115 wait complete. 2024-04-23T18:31:27.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 539451 3.61 MB/sec execute 452 sec latency 2919.559 ms 2024-04-23T18:31:28.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:28 smithi172 ceph-mon[32456]: pgmap v344: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-04-23T18:31:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:28 smithi173 ceph-mon[32906]: pgmap v344: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-04-23T18:31:28.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:28 smithi037 ceph-mon[29404]: pgmap v344: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-04-23T18:31:28.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541038 3.62 MB/sec execute 453 sec latency 14.212 ms 2024-04-23T18:31:29.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541671 3.62 MB/sec execute 454 sec latency 603.971 ms 2024-04-23T18:31:30.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:30 smithi172 ceph-mon[32456]: pgmap v345: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T18:31:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:30 smithi173 ceph-mon[32906]: pgmap v345: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T18:31:30.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:30 smithi037 ceph-mon[29404]: pgmap v345: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T18:31:30.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541671 3.61 MB/sec execute 455 sec latency 1604.081 ms 2024-04-23T18:31:31.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541671 3.61 MB/sec execute 456 sec latency 2604.212 ms 2024-04-23T18:31:32.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:32 smithi172 ceph-mon[32456]: pgmap v346: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T18:31:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:32 smithi173 ceph-mon[32906]: pgmap v346: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T18:31:32.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:32 smithi037 ceph-mon[29404]: pgmap v346: 97 pgs: 97 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T18:31:32.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 542420 3.60 MB/sec execute 457 sec latency 3337.218 ms 2024-04-23T18:31:33.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 544133 3.62 MB/sec execute 458 sec latency 18.397 ms 2024-04-23T18:31:34.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:34 smithi172 ceph-mon[32456]: pgmap v347: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-04-23T18:31:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:34 smithi173 ceph-mon[32906]: pgmap v347: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-04-23T18:31:34.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:34 smithi037 ceph-mon[29404]: pgmap v347: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-04-23T18:31:34.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 3.62 MB/sec execute 459 sec latency 362.658 ms 2024-04-23T18:31:35.849 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 3.62 MB/sec execute 460 sec latency 1362.702 ms 2024-04-23T18:31:36.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:36 smithi172 ceph-mon[32456]: pgmap v348: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-04-23T18:31:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:36 smithi173 ceph-mon[32906]: pgmap v348: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-04-23T18:31:36.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:36 smithi037 ceph-mon[29404]: pgmap v348: 97 pgs: 97 active+clean; 385 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-04-23T18:31:36.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 3.61 MB/sec execute 461 sec latency 2362.794 ms 2024-04-23T18:31:37.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545212 3.60 MB/sec execute 462 sec latency 3362.903 ms 2024-04-23T18:31:38.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:38 smithi172 ceph-mon[32456]: pgmap v349: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-04-23T18:31:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:38 smithi173 ceph-mon[32906]: pgmap v349: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-04-23T18:31:38.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:38 smithi037 ceph-mon[29404]: pgmap v349: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 162 op/s 2024-04-23T18:31:38.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 545975 3.60 MB/sec execute 463 sec latency 4072.670 ms 2024-04-23T18:31:39.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 547649 3.62 MB/sec execute 464 sec latency 24.154 ms 2024-04-23T18:31:40.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:40 smithi172 ceph-mon[32456]: pgmap v350: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-04-23T18:31:40.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:40 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:31:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:40 smithi173 ceph-mon[32906]: pgmap v350: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-04-23T18:31:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:40 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:31:40.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:40 smithi037 ceph-mon[29404]: pgmap v350: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-04-23T18:31:40.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:40 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 -- 172.21.15.37:0/4213183976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4968101aa0 msgr2=0x7f4968105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 --2- 172.21.15.37:0/4213183976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4968101aa0 0x7f4968105a90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4958009900 tx=0x7f495802a410 comp rx=0 tx=0).stop 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 -- 172.21.15.37:0/4213183976 shutdown_connections 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 --2- 172.21.15.37:0/4213183976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4968106170 0x7f496810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 --2- 172.21.15.37:0/4213183976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4968101aa0 0x7f4968105a90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 --2- 172.21.15.37:0/4213183976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49681010f0 0x7f49681014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 -- 172.21.15.37:0/4213183976 >> 172.21.15.37:0/4213183976 conn(0x7f4968077fd0 msgr2=0x7f49680783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 -- 172.21.15.37:0/4213183976 shutdown_connections 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.471+0000 7f4970364640 1 -- 172.21.15.37:0/4213183976 wait complete. 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 Processor -- start 2024-04-23T18:31:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 -- start start 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681010f0 0x7f4968198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4968106170 0x7f4968198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 0x7f496819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111090 con 0x7f496819cf70 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4968110f10 con 0x7f49681010f0 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f4970364640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4968111210 con 0x7f4968106170 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f496e901640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 0x7f496819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f496e100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681010f0 0x7f4968198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f496e901640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 0x7f496819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57428/0 (socket says 172.21.15.37:57428) 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f496e100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681010f0 0x7f4968198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:34156/0 (socket says 172.21.15.37:34156) 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.472+0000 7f496e901640 1 -- 172.21.15.37:0/3500419510 learned_addr learned my addr 172.21.15.37:0/3500419510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496d8ff640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4968106170 0x7f4968198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496e901640 1 -- 172.21.15.37:0/3500419510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4968106170 msgr2=0x7f4968198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496e901640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4968106170 0x7f4968198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496e901640 1 -- 172.21.15.37:0/3500419510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681010f0 msgr2=0x7f4968198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496e901640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681010f0 0x7f4968198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496e901640 1 -- 172.21.15.37:0/3500419510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496819da60 con 0x7f496819cf70 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f496e901640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 0x7f496819d420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4960009fd0 tx=0x7f496000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f495f7fe640 1 -- 172.21.15.37:0/3500419510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960017070 con 0x7f496819cf70 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496819dd50 con 0x7f496819cf70 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f495f7fe640 1 -- 172.21.15.37:0/3500419510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4960003e20 con 0x7f496819cf70 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f495f7fe640 1 -- 172.21.15.37:0/3500419510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960010460 con 0x7f496819cf70 2024-04-23T18:31:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.473+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a54c0 con 0x7f496819cf70 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.475+0000 7f495f7fe640 1 -- 172.21.15.37:0/3500419510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4960003970 con 0x7f496819cf70 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 Processor -- start 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 -- start start 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f066f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f067e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0681000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608f05d4970 con 0x5608f067ec10 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5608f05714a0 con 0x5608f066f540 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.476+0000 7f497307e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608f05d0930 con 0x5608f066fe60 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f495f7fe640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f49540a3dc0 0x7f49540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f066f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496e100640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f067e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f066f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:34160/0 (socket says 172.21.15.37:34160) 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 -- 172.21.15.37:0/2892699516 learned_addr learned my addr 172.21.15.37:0/2892699516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0681000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496e901640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f49540a3dc0 0x7f49540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f495f7fe640 1 -- 172.21.15.37:0/3500419510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f49600d05a0 con 0x7f496819cf70 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 -- 172.21.15.37:0/2892699516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 msgr2=0x5608f067e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f067e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 -- 172.21.15.37:0/2892699516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 msgr2=0x5608f0681000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0681000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 -- 172.21.15.37:0/2892699516 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f02e7210 con 0x5608f066f540 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496d8ff640 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f066f920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4958005c40 tx=0x7f49580058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24698a1c882f2a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f496e901640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f49540a3dc0 0x7f49540a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4960098940 tx=0x7f4960004020 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.477+0000 7f495cbfe640 1 -- 172.21.15.37:0/2892699516 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958035040 con 0x5608f066f540 2024-04-23T18:31:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f495cbfe640 1 -- 172.21.15.37:0/2892699516 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49580035f0 con 0x5608f066f540 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f495cbfe640 1 -- 172.21.15.37:0/2892699516 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580044e0 con 0x5608f066f540 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 -- 172.21.15.37:0/2892699516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 msgr2=0x5608f066f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f066f920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4958005c40 tx=0x7f49580058d0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 -- 172.21.15.37:0/2892699516 shutdown_connections 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0681000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f067e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 --2- 172.21.15.37:0/2892699516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f066f920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 -- 172.21.15.37:0/2892699516 >> 172.21.15.37:0/2892699516 conn(0x5608f066bb30 msgr2=0x5608f066bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 -- 172.21.15.37:0/2892699516 shutdown_connections 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.478+0000 7f497307e740 1 -- 172.21.15.37:0/2892699516 wait complete. 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 -- start start 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 Processor -- start 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f067e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f067b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608f06817e0 con 0x5608f066f540 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5608f056f0b0 con 0x5608f066fe60 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608f0681960 con 0x5608f067ec10 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f497307e740 10 client.0 _ll_register_callbacks cb 0x5608f055cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f067e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f067e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57442/0 (socket says 172.21.15.37:57442) 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f496d8ff640 1 -- 172.21.15.37:0/1001837301 learned_addr learned my addr 172.21.15.37:0/1001837301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f496e100640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f067b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.479+0000 7f496d8ff640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f496d8ff640 1 -- 172.21.15.37:0/1001837301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 msgr2=0x5608f067be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f496d8ff640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f496d8ff640 1 -- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 msgr2=0x5608f067b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f496d8ff640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f067b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f496d8ff640 1 -- 172.21.15.37:0/1001837301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f03b3620 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f496d8ff640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f067e540 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f495802a450 tx=0x7f4958009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bb0f710ef99c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f4951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f4951ffb640 1 -- 172.21.15.37:0/1001837301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958035070 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f4951ffb640 1 -- 172.21.15.37:0/1001837301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f49580039a0 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49580035f0 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f4951ffb640 1 -- 172.21.15.37:0/1001837301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4958003ba0 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f497307e740 10 client.16749 fetch_fsmap learned FSMap version 43 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5608f068d750 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f4951ffb640 1 -- 172.21.15.37:0/1001837301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49580035f0 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f4951ffb640 1 -- 172.21.15.37:0/1001837301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f495803a040 con 0x5608f066f540 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f497307e740 10 client.16749 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:31:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.480+0000 7f497307e740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f497307e740 20 client.16749 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f497307e740 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f068eaa0 0x5608f0690e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f497307e740 4 client.16749 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f05e2390 con 0x5608f068eaa0 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f496e100640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f068eaa0 0x5608f0690e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f496e100640 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f068eaa0 0x5608f0690e90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6869a59bbe4a24cb server_cookie=f8f90df1ea24af79 in_seq=0 out_seq=0 2024-04-23T18:31:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.481+0000 7f4951ffb640 10 client.16749 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f4951ffb640 1 -- 172.21.15.37:0/1001837301 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608f05e2390 con 0x5608f068eaa0 2024-04-23T18:31:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f4951ffb640 10 client.16749 handle_command_reply: tid=0 2024-04-23T18:31:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 client.16749 shutdown 2024-04-23T18:31:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 msgr2=0x5608f067e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f067e540 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f495802a450 tx=0x7f4958009560 comp rx=0 tx=0).stop 2024-04-23T18:31:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 shutdown_connections 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f068eaa0 0x5608f0690e90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f067b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f497307e740 1 --2- 172.21.15.37:0/1001837301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f067e540 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.482+0000 7f4951ffb640 0 client.16749 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.483+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 >> 172.21.15.37:0/1001837301 conn(0x5608f066bb30 msgr2=0x5608f06719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.483+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 shutdown_connections 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.483+0000 7f497307e740 1 -- 172.21.15.37:0/1001837301 wait complete. 2024-04-23T18:31:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.483+0000 7f497307e740 20 client.16749 trim_cache size 0 max 16384 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.497+0000 7f497307e740 1 Processor -- start 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.497+0000 7f497307e740 1 -- start start 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f497307e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f068a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f497307e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f066ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f497307e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f497307e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49681ae170 con 0x5608f066f540 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f497307e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49681aba10 con 0x5608f066fe60 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f497307e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49681a9880 con 0x5608f067ec10 2024-04-23T18:31:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f068a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f068a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57452/0 (socket says 172.21.15.37:57452) 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 -- 172.21.15.37:0/65783815 learned_addr learned my addr 172.21.15.37:0/65783815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 -- 172.21.15.37:0/65783815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 msgr2=0x5608f067e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 -- 172.21.15.37:0/65783815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 msgr2=0x5608f066ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f066ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 -- 172.21.15.37:0/65783815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f067c1b0 con 0x5608f066f540 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f496d8ff640 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f068a6a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4958005c90 tx=0x7f4958005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79d88651057d551 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f4953fff640 1 -- 172.21.15.37:0/65783815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958035070 con 0x5608f066f540 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f4953fff640 1 -- 172.21.15.37:0/65783815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4958003d40 con 0x5608f066f540 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.498+0000 7f4953fff640 1 -- 172.21.15.37:0/65783815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958003a00 con 0x5608f066f540 2024-04-23T18:31:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- 172.21.15.37:0/65783815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 msgr2=0x5608f068a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f068a6a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4958005c90 tx=0x7f4958005cc0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- 172.21.15.37:0/65783815 shutdown_connections 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f067ec10 0x5608f067e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066fe60 0x5608f066ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- 172.21.15.37:0/65783815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f066f540 0x5608f068a6a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- 172.21.15.37:0/65783815 >> 172.21.15.37:0/65783815 conn(0x5608f066bb30 msgr2=0x5608f066a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- 172.21.15.37:0/65783815 shutdown_connections 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- 172.21.15.37:0/65783815 wait complete. 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- start start 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 Processor -- start 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f0687120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f06783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0678a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49681a9ed0 con 0x5608f067ec10 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49681ab340 con 0x5608f066f540 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49681aa5a0 con 0x5608f066fe60 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.499+0000 7f497307e740 10 client.0 _ll_register_callbacks cb 0x7f49580316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0678a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0678a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57454/0 (socket says 172.21.15.37:57454) 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 -- 172.21.15.37:0/418032448 learned_addr learned my addr 172.21.15.37:0/418032448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496e100640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f06783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f0687120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 -- 172.21.15.37:0/418032448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 msgr2=0x5608f06783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f06783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 -- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 msgr2=0x5608f0687120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f0687120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 -- 172.21.15.37:0/418032448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f06f3d00 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496e100640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f06783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f496d8ff640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0678a90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f495802a930 tx=0x7f49580038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c656bbcbb74516b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f4951ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f4951ffb640 1 -- 172.21.15.37:0/418032448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958035030 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f4951ffb640 1 -- 172.21.15.37:0/418032448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f4958056450 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4958003d40 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f4951ffb640 1 -- 172.21.15.37:0/418032448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f49580039e0 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f497307e740 10 client.16755 fetch_fsmap learned FSMap version 43 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.500+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5608f06f4eb0 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f4951ffb640 1 -- 172.21.15.37:0/418032448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4958003d40 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f4951ffb640 1 -- 172.21.15.37:0/418032448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f4958066030 con 0x5608f067ec10 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f497307e740 10 client.16755 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f497307e740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f497307e740 20 client.16755 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f497307e740 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f06f5280 0x5608f06f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f497307e740 4 client.16755 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608f05dc200 con 0x5608f06f5280 2024-04-23T18:31:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f496e100640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f06f5280 0x5608f06f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f4951ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.501+0000 7f496e100640 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f06f5280 0x5608f06f76a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afaa9b2994b9d7e0 server_cookie=688baf8a33360f5f in_seq=0 out_seq=0 2024-04-23T18:31:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.542+0000 7f4951ffb640 1 -- 172.21.15.37:0/418032448 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608f05dc200 con 0x5608f06f5280 2024-04-23T18:31:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.542+0000 7f4951ffb640 10 client.16755 handle_command_reply: tid=0 2024-04-23T18:31:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.542+0000 7f497307e740 1 client.16755 shutdown 2024-04-23T18:31:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.542+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 msgr2=0x5608f0678a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.542+0000 7f497307e740 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0678a90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f495802a930 tx=0x7f49580038e0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.542+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 shutdown_connections 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5608f06f5280 0x5608f06f76a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f067ec10 0x5608f0678a90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608f066fe60 0x5608f06783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4951ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 --2- 172.21.15.37:0/418032448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608f066f540 0x5608f0687120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 >> 172.21.15.37:0/418032448 conn(0x5608f066bb30 msgr2=0x5608f0676f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 shutdown_connections 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 1 -- 172.21.15.37:0/418032448 wait complete. 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f497307e740 20 client.16755 trim_cache size 0 max 16384 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:31:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:31:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:31:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:31:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f49540a3dc0 msgr2=0x7f49540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f49540a3dc0 0x7f49540a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4960098940 tx=0x7f4960004020 comp rx=0 tx=0).stop 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 msgr2=0x7f496819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 0x7f496819d420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4960009fd0 tx=0x7f496000b260 comp rx=0 tx=0).stop 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 shutdown_connections 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f49540a3dc0 0x7f49540a6280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496819cf70 0x7f496819d420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4968106170 0x7f4968198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 --2- 172.21.15.37:0/3500419510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681010f0 0x7f4968198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 >> 172.21.15.37:0/3500419510 conn(0x7f4968077fd0 msgr2=0x7f4968104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.543+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 shutdown_connections 2024-04-23T18:31:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:40.544+0000 7f4970364640 1 -- 172.21.15.37:0/3500419510 wait complete. 2024-04-23T18:31:40.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 549652 3.63 MB/sec execute 465 sec latency 58.125 ms 2024-04-23T18:31:41.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 551394 3.64 MB/sec execute 466 sec latency 18.259 ms 2024-04-23T18:31:42.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:42 smithi172 ceph-mon[32456]: pgmap v351: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-04-23T18:31:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:42 smithi173 ceph-mon[32906]: pgmap v351: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-04-23T18:31:42.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:42 smithi037 ceph-mon[29404]: pgmap v351: 97 pgs: 97 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-04-23T18:31:42.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 553063 3.65 MB/sec execute 467 sec latency 211.460 ms 2024-04-23T18:31:43.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 554698 3.67 MB/sec execute 468 sec latency 24.008 ms 2024-04-23T18:31:44.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:44 smithi172 ceph-mon[32456]: pgmap v352: 97 pgs: 97 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-04-23T18:31:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:44 smithi173 ceph-mon[32906]: pgmap v352: 97 pgs: 97 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-04-23T18:31:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:44 smithi037 ceph-mon[29404]: pgmap v352: 97 pgs: 97 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-04-23T18:31:44.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 555852 3.67 MB/sec execute 469 sec latency 270.475 ms 2024-04-23T18:31:45.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 555852 3.66 MB/sec execute 470 sec latency 1270.526 ms 2024-04-23T18:31:46.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:46 smithi172 ceph-mon[32456]: pgmap v353: 97 pgs: 97 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 136 op/s 2024-04-23T18:31:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:46 smithi173 ceph-mon[32906]: pgmap v353: 97 pgs: 97 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 136 op/s 2024-04-23T18:31:46.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:46 smithi037 ceph-mon[29404]: pgmap v353: 97 pgs: 97 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.9 MiB/s wr, 136 op/s 2024-04-23T18:31:46.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 555852 3.65 MB/sec execute 471 sec latency 2270.573 ms 2024-04-23T18:31:47.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:47 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:47.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:47 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:47.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:47 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:47.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 555852 3.65 MB/sec execute 472 sec latency 3270.626 ms 2024-04-23T18:31:48.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:48 smithi172 ceph-mon[32456]: pgmap v354: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-04-23T18:31:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:48 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:48 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:48 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:48 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:48.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:48 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:48 smithi173 ceph-mon[32906]: pgmap v354: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-04-23T18:31:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:48 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:48 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:48 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:48 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:48 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:48 smithi037 ceph-mon[29404]: pgmap v354: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-04-23T18:31:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:48 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:48 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:48 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:48 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:48.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:48 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:31:48.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 556639 3.64 MB/sec execute 473 sec latency 3974.042 ms 2024-04-23T18:31:49.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 558325 3.66 MB/sec execute 474 sec latency 22.187 ms 2024-04-23T18:31:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:50 smithi172 ceph-mon[32456]: pgmap v355: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-04-23T18:31:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:50 smithi173 ceph-mon[32906]: pgmap v355: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-04-23T18:31:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:50 smithi037 ceph-mon[29404]: pgmap v355: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-04-23T18:31:50.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559397 3.66 MB/sec execute 475 sec latency 429.819 ms 2024-04-23T18:31:51.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559397 3.66 MB/sec execute 476 sec latency 1429.901 ms 2024-04-23T18:31:52.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:52 smithi172 ceph-mon[32456]: pgmap v356: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-04-23T18:31:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:52 smithi173 ceph-mon[32906]: pgmap v356: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-04-23T18:31:52.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:52 smithi037 ceph-mon[29404]: pgmap v356: 97 pgs: 97 active+clean; 401 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-04-23T18:31:52.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 560135 3.65 MB/sec execute 477 sec latency 2154.835 ms 2024-04-23T18:31:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.634+0000 7f238fbe1640 1 -- 172.21.15.37:0/1517756976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23881072b0 msgr2=0x7f238810ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.634+0000 7f238fbe1640 1 --2- 172.21.15.37:0/1517756976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23881072b0 0x7f238810ee80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2380009ca0 tx=0x7f238002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 -- 172.21.15.37:0/1517756976 shutdown_connections 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 --2- 172.21.15.37:0/1517756976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23881072b0 0x7f238810ee80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 --2- 172.21.15.37:0/1517756976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2388102b90 0x7f2388106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 --2- 172.21.15.37:0/1517756976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23881021e0 0x7f23881025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 -- 172.21.15.37:0/1517756976 >> 172.21.15.37:0/1517756976 conn(0x7f23880fda00 msgr2=0x7f23880ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 -- 172.21.15.37:0/1517756976 shutdown_connections 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 -- 172.21.15.37:0/1517756976 wait complete. 2024-04-23T18:31:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 Processor -- start 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 -- start start 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.635+0000 7f238fbe1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23881021e0 0x7f2388198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23881072b0 0x7f2388198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 0x7f238819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2388112220 con 0x7f238819cef0 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23881120a0 con 0x7f23881021e0 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23881123a0 con 0x7f23881072b0 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 0x7f238819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238d97d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23881021e0 0x7f2388198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 0x7f238819d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34520/0 (socket says 172.21.15.37:34520) 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 -- 172.21.15.37:0/613675250 learned_addr learned my addr 172.21.15.37:0/613675250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238d17c640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23881072b0 0x7f2388198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 -- 172.21.15.37:0/613675250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23881072b0 msgr2=0x7f2388198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23881072b0 0x7f2388198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 -- 172.21.15.37:0/613675250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23881021e0 msgr2=0x7f2388198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23881021e0 0x7f2388198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 -- 172.21.15.37:0/613675250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238819d9e0 con 0x7f238819cef0 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238e17e640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 0x7f238819d3a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f23800098d0 tx=0x7f23800317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f237effd640 1 -- 172.21.15.37:0/613675250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380003b70 con 0x7f238819cef0 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f237effd640 1 -- 172.21.15.37:0/613675250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2380003d10 con 0x7f238819cef0 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238819dc70 con 0x7f238819cef0 2024-04-23T18:31:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f237effd640 1 -- 172.21.15.37:0/613675250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238002f9d0 con 0x7f238819cef0 2024-04-23T18:31:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.636+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881a54c0 con 0x7f238819cef0 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.639+0000 7f237effd640 1 -- 172.21.15.37:0/613675250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f238002fb90 con 0x7f238819cef0 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.639+0000 7f23929cc740 1 Processor -- start 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.639+0000 7f23929cc740 1 -- start start 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f23929cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67da93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f23929cc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 0x55b67da9e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f23929cc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67daa0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f23929cc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b67d9f8970 con 0x55b67da93f40 2024-04-23T18:31:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f23929cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b67d9954a0 con 0x55b67da93620 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f23929cc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b67d9f4930 con 0x55b67da9e800 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 0x55b67da9e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 0x55b67da9e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34534/0 (socket says 172.21.15.37:34534) 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d17c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67da93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 -- 172.21.15.37:0/4184441062 learned_addr learned my addr 172.21.15.37:0/4184441062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d17c640 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67daa0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f237effd640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f23740a3e40 0x7f23740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 -- 172.21.15.37:0/4184441062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 msgr2=0x55b67daa0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67daa0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 -- 172.21.15.37:0/4184441062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 msgr2=0x55b67da93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67da93a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 -- 172.21.15.37:0/4184441062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67d70b210 con 0x55b67da93f40 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d17c640 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67da93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238e17e640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f23740a3e40 0x7f23740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f237effd640 1 -- 172.21.15.37:0/613675250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f23800f1aa0 con 0x7f238819cef0 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.640+0000 7f238d97d640 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 0x55b67da9e2c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f238400ad00 tx=0x7f238400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893a0ce75f6489b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.641+0000 7f2373fff640 1 -- 172.21.15.37:0/4184441062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238401d070 con 0x55b67da93f40 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.641+0000 7f2373fff640 1 -- 172.21.15.37:0/4184441062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2384018af0 con 0x55b67da93f40 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.641+0000 7f2373fff640 1 -- 172.21.15.37:0/4184441062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384010a70 con 0x55b67da93f40 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f238e17e640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f23740a3e40 0x7f23740a6300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f23800ef370 tx=0x7f23800de040 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 -- 172.21.15.37:0/4184441062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 msgr2=0x55b67da9e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 0x55b67da9e2c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f238400ad00 tx=0x7f238400f790 comp rx=0 tx=0).stop 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 -- 172.21.15.37:0/4184441062 shutdown_connections 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67daa0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93f40 0x55b67da9e2c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 --2- 172.21.15.37:0/4184441062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67da93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 -- 172.21.15.37:0/4184441062 >> 172.21.15.37:0/4184441062 conn(0x55b67da8fb30 msgr2=0x55b67da8ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 -- 172.21.15.37:0/4184441062 shutdown_connections 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 -- 172.21.15.37:0/4184441062 wait complete. 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 -- start start 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.642+0000 7f23929cc740 1 Processor -- start 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67da9e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da9b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da9b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b67daa1430 con 0x55b67da93620 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b67d9930b0 con 0x55b67da93f40 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b67daa15b0 con 0x55b67da9e800 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23929cc740 10 client.0 _ll_register_callbacks cb 0x55b67d980d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67da9e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67da9e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34540/0 (socket says 172.21.15.37:34540) 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 -- 172.21.15.37:0/1795759593 learned_addr learned my addr 172.21.15.37:0/1795759593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d97d640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da9b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da9b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 -- 172.21.15.37:0/1795759593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 msgr2=0x55b67da9b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da9b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 -- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 msgr2=0x55b67da9b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da9b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 -- 172.21.15.37:0/1795759593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67d7d7620 con 0x55b67da93620 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d97d640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da9b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f238d17c640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67da9e170 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2378004640 tx=0x7f2378004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda7d7c455f14fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23717fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.643+0000 7f23717fa640 1 -- 172.21.15.37:0/1795759593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378025060 con 0x55b67da93620 2024-04-23T18:31:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2384018af0 con 0x55b67da93620 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23717fa640 1 -- 172.21.15.37:0/1795759593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f2378012ab0 con 0x55b67da93620 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23717fa640 1 -- 172.21.15.37:0/1795759593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f237801a040 con 0x55b67da93620 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 10 client.16767 fetch_fsmap learned FSMap version 43 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b67daad450 con 0x55b67da93620 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23717fa640 1 -- 172.21.15.37:0/1795759593 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2384018af0 con 0x55b67da93620 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23717fa640 1 -- 172.21.15.37:0/1795759593 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f2378020ba0 con 0x55b67da93620 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 10 client.16767 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 20 client.16767 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67daae8c0 0x55b67dab0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 4 client.16767 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b67da06390 con 0x55b67daae8c0 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.644+0000 7f238d97d640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67daae8c0 0x55b67dab0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f23717fa640 10 client.16767 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f238d97d640 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67daae8c0 0x55b67dab0cb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e841acfac0d5db server_cookie=f4c87c9cd9bc299b in_seq=0 out_seq=0 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f23717fa640 1 -- 172.21.15.37:0/1795759593 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b67da06390 con 0x55b67daae8c0 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f23717fa640 10 client.16767 handle_command_reply: tid=0 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f23929cc740 1 client.16767 shutdown 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 msgr2=0x55b67da9e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.645+0000 7f23929cc740 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67da9e170 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2378004640 tx=0x7f2378004730 comp rx=0 tx=0).stop 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 shutdown_connections 2024-04-23T18:31:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67daae8c0 0x55b67dab0cb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da9b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da9b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23717fa640 0 client.16767 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 --2- 172.21.15.37:0/1795759593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67da9e170 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 >> 172.21.15.37:0/1795759593 conn(0x55b67da8fb30 msgr2=0x55b67da958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 shutdown_connections 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 1 -- 172.21.15.37:0/1795759593 wait complete. 2024-04-23T18:31:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.646+0000 7f23929cc740 20 client.16767 trim_cache size 0 max 16384 2024-04-23T18:31:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 Processor -- start 2024-04-23T18:31:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 -- start start 2024-04-23T18:31:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67daaa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da93f40 0x55b67da9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 0x55b67da9dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23881b04c0 con 0x55b67da9e800 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23881add60 con 0x55b67da93620 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f23929cc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23881abbd0 con 0x55b67da93f40 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f238d17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 0x55b67da9dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f238d17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 0x55b67da9dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34550/0 (socket says 172.21.15.37:34550) 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f238d17c640 1 -- 172.21.15.37:0/638948358 learned_addr learned my addr 172.21.15.37:0/638948358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f238d17c640 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67daaa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.661+0000 7f238d97d640 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da93f40 0x55b67da9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f238d17c640 1 -- 172.21.15.37:0/638948358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da93f40 msgr2=0x55b67da9a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f238d17c640 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da93f40 0x55b67da9a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f238d17c640 1 -- 172.21.15.37:0/638948358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 msgr2=0x55b67daaa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f238d17c640 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67daaa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f238d17c640 1 -- 172.21.15.37:0/638948358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67da93d70 con 0x55b67da9e800 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f238d17c640 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 0x55b67da9dca0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2378003720 tx=0x7f23780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af77523ec796c463 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f23737fe640 1 -- 172.21.15.37:0/638948358 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378012b40 con 0x55b67da9e800 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f23737fe640 1 -- 172.21.15.37:0/638948358 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378012ce0 con 0x55b67da9e800 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.662+0000 7f23737fe640 1 -- 172.21.15.37:0/638948358 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378020650 con 0x55b67da9e800 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 -- 172.21.15.37:0/638948358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 msgr2=0x55b67da9dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 0x55b67da9dca0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2378003720 tx=0x7f23780046c0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 -- 172.21.15.37:0/638948358 shutdown_connections 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da9e800 0x55b67da9dca0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da93f40 0x55b67da9a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 --2- 172.21.15.37:0/638948358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93620 0x55b67daaa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 -- 172.21.15.37:0/638948358 >> 172.21.15.37:0/638948358 conn(0x55b67da8fb30 msgr2=0x55b67da96af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 -- 172.21.15.37:0/638948358 shutdown_connections 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 -- 172.21.15.37:0/638948358 wait complete. 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 -- start start 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 Processor -- start 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.663+0000 7f23929cc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67daa6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da97df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da98330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ac220 con 0x55b67da93620 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ad690 con 0x55b67da93f40 2024-04-23T18:31:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ac8f0 con 0x55b67da9e800 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 10 client.0 _ll_register_callbacks cb 0x7f2378004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67daa6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67daa6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34564/0 (socket says 172.21.15.37:34564) 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 -- 172.21.15.37:0/2436429359 learned_addr learned my addr 172.21.15.37:0/2436429359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d97d640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da97df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da98330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 -- 172.21.15.37:0/2436429359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 msgr2=0x55b67da98330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da98330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 -- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 msgr2=0x55b67da97df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da97df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 -- 172.21.15.37:0/2436429359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67da98a70 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d97d640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da97df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f238d17c640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67daa6af0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2378004440 tx=0x7f2378009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5942d1d57eb24a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23717fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23717fa640 1 -- 172.21.15.37:0/2436429359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378025070 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23717fa640 1 -- 172.21.15.37:0/2436429359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f237800f990 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2378012ce0 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23717fa640 1 -- 172.21.15.37:0/2436429359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2378044050 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 10 client.16773 fetch_fsmap learned FSMap version 43 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.664+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b67db14db0 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23717fa640 1 -- 172.21.15.37:0/2436429359 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378012ce0 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23717fa640 1 -- 172.21.15.37:0/2436429359 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f2378035440 con 0x55b67da93620 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23929cc740 10 client.16773 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:31:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23929cc740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23929cc740 20 client.16773 populate_metadata read hostname 'smithi037' 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23929cc740 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67db15420 0x55b67db17810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23929cc740 4 client.16773 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b67da00200 con 0x55b67db15420 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.665+0000 7f238d97d640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67db15420 0x55b67db17810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.666+0000 7f23717fa640 10 client.16773 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.666+0000 7f238d97d640 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67db15420 0x55b67db17810 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722ef590893daaa3 server_cookie=72cdf26512937e33 in_seq=0 out_seq=0 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23717fa640 1 -- 172.21.15.37:0/2436429359 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b67da00200 con 0x55b67db15420 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23717fa640 10 client.16773 handle_command_reply: tid=0 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 client.16773 shutdown 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 msgr2=0x55b67daa6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67daa6af0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2378004440 tx=0x7f2378009ce0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 shutdown_connections 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55b67db15420 0x55b67db17810 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b67da9e800 0x55b67da98330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67da93f40 0x55b67da97df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 --2- 172.21.15.37:0/2436429359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b67da93620 0x55b67daa6af0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 >> 172.21.15.37:0/2436429359 conn(0x55b67da8fb30 msgr2=0x55b67da95020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23717fa640 0 client.16773 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 shutdown_connections 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 1 -- 172.21.15.37:0/2436429359 wait complete. 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.699+0000 7f23929cc740 20 client.16773 trim_cache size 0 max 16384 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:31:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:31:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f23740a3e40 msgr2=0x7f23740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f23740a3e40 0x7f23740a6300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f23800ef370 tx=0x7f23800de040 comp rx=0 tx=0).stop 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 msgr2=0x7f238819d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 0x7f238819d3a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f23800098d0 tx=0x7f23800317b0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 shutdown_connections 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f23740a3e40 0x7f23740a6300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f238819cef0 0x7f238819d3a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23881072b0 0x7f2388198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 --2- 172.21.15.37:0/613675250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23881021e0 0x7f2388198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 >> 172.21.15.37:0/613675250 conn(0x7f23880fda00 msgr2=0x7f23880fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 shutdown_connections 2024-04-23T18:31:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:31:53.700+0000 7f238fbe1640 1 -- 172.21.15.37:0/613675250 wait complete. 2024-04-23T18:31:53.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 561827 3.67 MB/sec execute 478 sec latency 25.652 ms 2024-04-23T18:31:54.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:54 smithi172 ceph-mon[32456]: pgmap v357: 97 pgs: 97 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 190 op/s 2024-04-23T18:31:54.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:54 smithi173 ceph-mon[32906]: pgmap v357: 97 pgs: 97 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 190 op/s 2024-04-23T18:31:54.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:54 smithi037 ceph-mon[29404]: pgmap v357: 97 pgs: 97 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 190 op/s 2024-04-23T18:31:54.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 562946 3.67 MB/sec execute 479 sec latency 351.509 ms 2024-04-23T18:31:55.850 INFO:tasks.workunit.client.0.smithi037.stdout: 1 562946 3.67 MB/sec execute 480 sec latency 1351.587 ms 2024-04-23T18:31:56.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:56 smithi172 ceph-mon[32456]: pgmap v358: 97 pgs: 97 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-23T18:31:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:56 smithi173 ceph-mon[32906]: pgmap v358: 97 pgs: 97 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-23T18:31:56.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:56 smithi037 ceph-mon[29404]: pgmap v358: 97 pgs: 97 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-04-23T18:31:56.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 562946 3.66 MB/sec execute 481 sec latency 2351.709 ms 2024-04-23T18:31:57.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 562946 3.65 MB/sec execute 482 sec latency 3351.850 ms 2024-04-23T18:31:58.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:31:58 smithi172 ceph-mon[32456]: pgmap v359: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-04-23T18:31:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:31:58 smithi173 ceph-mon[32906]: pgmap v359: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-04-23T18:31:58.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:31:58 smithi037 ceph-mon[29404]: pgmap v359: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-04-23T18:31:58.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 563696 3.65 MB/sec execute 483 sec latency 4061.554 ms 2024-04-23T18:31:59.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 565356 3.66 MB/sec execute 484 sec latency 12.505 ms 2024-04-23T18:32:00.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:00 smithi172 ceph-mon[32456]: pgmap v360: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-23T18:32:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:00 smithi173 ceph-mon[32906]: pgmap v360: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-23T18:32:00.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:00 smithi037 ceph-mon[29404]: pgmap v360: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-23T18:32:00.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 566179 3.67 MB/sec execute 485 sec latency 167.896 ms 2024-04-23T18:32:01.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 566486 3.66 MB/sec execute 486 sec latency 784.934 ms 2024-04-23T18:32:02.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:02 smithi172 ceph-mon[32456]: pgmap v361: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-23T18:32:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:02 smithi173 ceph-mon[32906]: pgmap v361: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-23T18:32:02.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:02 smithi037 ceph-mon[29404]: pgmap v361: 97 pgs: 97 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-23T18:32:02.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567203 3.66 MB/sec execute 487 sec latency 1514.903 ms 2024-04-23T18:32:03.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 568869 3.67 MB/sec execute 488 sec latency 12.791 ms 2024-04-23T18:32:04.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:04 smithi172 ceph-mon[32456]: pgmap v362: 97 pgs: 97 active+clean; 414 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-04-23T18:32:04.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:04 smithi173 ceph-mon[32906]: pgmap v362: 97 pgs: 97 active+clean; 414 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-04-23T18:32:04.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:04 smithi037 ceph-mon[29404]: pgmap v362: 97 pgs: 97 active+clean; 414 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-04-23T18:32:04.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 570036 3.68 MB/sec execute 489 sec latency 199.793 ms 2024-04-23T18:32:05.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:05 smithi172 ceph-mon[32456]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:32:05.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:05 smithi173 ceph-mon[32906]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:32:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:05 smithi037 ceph-mon[29404]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:32:05.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 570036 3.67 MB/sec execute 490 sec latency 1199.914 ms 2024-04-23T18:32:06.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:06 smithi172 ceph-mon[32456]: pgmap v364: 97 pgs: 97 active+clean; 414 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-04-23T18:32:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:06 smithi173 ceph-mon[32906]: pgmap v364: 97 pgs: 97 active+clean; 414 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-04-23T18:32:06.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:06 smithi037 ceph-mon[29404]: pgmap v364: 97 pgs: 97 active+clean; 414 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-04-23T18:32:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.799+0000 7f727ea72640 1 -- 172.21.15.37:0/2193436307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781010e0 msgr2=0x7f72781014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.799+0000 7f727ea72640 1 --2- 172.21.15.37:0/2193436307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781010e0 0x7f72781014c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 -- 172.21.15.37:0/2193436307 shutdown_connections 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 --2- 172.21.15.37:0/2193436307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781061b0 0x7f727810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 --2- 172.21.15.37:0/2193436307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7278101a90 0x7f7278105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 --2- 172.21.15.37:0/2193436307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781010e0 0x7f72781014c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 -- 172.21.15.37:0/2193436307 >> 172.21.15.37:0/2193436307 conn(0x7f72780771d0 msgr2=0x7f72780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 -- 172.21.15.37:0/2193436307 shutdown_connections 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 -- 172.21.15.37:0/2193436307 wait complete. 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 Processor -- start 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 -- start start 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.800+0000 7f727ea72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781010e0 0x7f7278198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727ea72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 0x7f7278198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727ea72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f727819cf40 0x7f727819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727ea72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f72781110f0 con 0x7f72781061b0 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727ea72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7278110f70 con 0x7f727819cf40 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727ea72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7278111270 con 0x7f72781010e0 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 0x7f7278198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 0x7f7278198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51438/0 (socket says 172.21.15.37:51438) 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 -- 172.21.15.37:0/2428672986 learned_addr learned my addr 172.21.15.37:0/2428672986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727d00f640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f727819cf40 0x7f727819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727c80e640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781010e0 0x7f7278198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 -- 172.21.15.37:0/2428672986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781010e0 msgr2=0x7f7278198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781010e0 0x7f7278198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 -- 172.21.15.37:0/2428672986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f727819cf40 msgr2=0x7f727819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f727819cf40 0x7f727819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 -- 172.21.15.37:0/2428672986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727819da30 con 0x7f72781061b0 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727d00f640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f727819cf40 0x7f727819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727c80e640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781010e0 0x7f7278198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7277fff640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 0x7f7278198bc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f726400d770 tx=0x7f726400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7275ffb640 1 -- 172.21.15.37:0/2428672986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264017070 con 0x7f72781061b0 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7275ffb640 1 -- 172.21.15.37:0/2428672986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264009c30 con 0x7f72781061b0 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727819dd20 con 0x7f72781061b0 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.801+0000 7f7275ffb640 1 -- 172.21.15.37:0/2428672986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726400f620 con 0x7f72781061b0 2024-04-23T18:32:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.802+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72781a5510 con 0x7f72781061b0 2024-04-23T18:32:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7275ffb640 1 -- 172.21.15.37:0/2428672986 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f726401a070 con 0x7f72781061b0 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 Processor -- start 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 -- start start 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 0x55a05c5fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5fb800 0x55a05c5fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a05c555970 con 0x55a05c5f0f40 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a05c4f24a0 con 0x55a05c5f0620 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7281783740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a05c551930 con 0x55a05c5fb800 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f727c80e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 0x55a05c5fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7275ffb640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f72680a3dc0 0x7f72680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f727c80e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 0x55a05c5fb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51450/0 (socket says 172.21.15.37:51450) 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f727c80e640 1 -- 172.21.15.37:0/1291641266 learned_addr learned my addr 172.21.15.37:0/1291641266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.805+0000 7f7277fff640 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5fb800 0x55a05c5fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727d00f640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f72680a3dc0 0x7f72680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727d00f640 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f7275ffb640 1 -- 172.21.15.37:0/2428672986 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8623+0+0 (secure 0 0 0) 0x7f72640d0480 con 0x7f72781061b0 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727c80e640 1 -- 172.21.15.37:0/1291641266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5fb800 msgr2=0x55a05c5fdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727c80e640 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5fb800 0x55a05c5fdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727c80e640 1 -- 172.21.15.37:0/1291641266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 msgr2=0x55a05c5f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727c80e640 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727c80e640 1 -- 172.21.15.37:0/1291641266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05c268210 con 0x55a05c5f0f40 2024-04-23T18:32:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727c80e640 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 0x55a05c5fb2c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1e79a679133260 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f727d00f640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f72680a3dc0 0x7f72680a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7270009370 tx=0x7f7270009570 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f7262ffd640 1 -- 172.21.15.37:0/1291641266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c03f070 con 0x55a05c5f0f40 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f7262ffd640 1 -- 172.21.15.37:0/1291641266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c0316a0 con 0x55a05c5f0f40 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.806+0000 7f7262ffd640 1 -- 172.21.15.37:0/1291641266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c035070 con 0x55a05c5f0f40 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 -- 172.21.15.37:0/1291641266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 msgr2=0x55a05c5fb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 0x55a05c5fb2c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c032cd0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 -- 172.21.15.37:0/1291641266 shutdown_connections 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5fb800 0x55a05c5fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5f0f40 0x55a05c5fb2c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 --2- 172.21.15.37:0/1291641266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 -- 172.21.15.37:0/1291641266 >> 172.21.15.37:0/1291641266 conn(0x55a05c5ecb30 msgr2=0x55a05c5ecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 -- 172.21.15.37:0/1291641266 shutdown_connections 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 -- 172.21.15.37:0/1291641266 wait complete. 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 -- start start 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.807+0000 7f7281783740 1 Processor -- start 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c5f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c5f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a05c5fe3d0 con 0x55a05c5fb800 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a05c4f00b0 con 0x55a05c5f0620 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a05c5fe550 con 0x55a05c5f0f40 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7281783740 10 client.0 _ll_register_callbacks cb 0x55a05c4ddd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c5f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c5f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51456/0 (socket says 172.21.15.37:51456) 2024-04-23T18:32:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 -- 172.21.15.37:0/907738512 learned_addr learned my addr 172.21.15.37:0/907738512 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f7277fff640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c5f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727c80e640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 -- 172.21.15.37:0/907738512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 msgr2=0x55a05c5f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c5f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 -- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 msgr2=0x55a05c5fb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5fb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727d00f640 1 -- 172.21.15.37:0/907738512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05c334620 con 0x55a05c5fb800 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.808+0000 7f727c80e640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f727d00f640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c5f8900 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f727000fa80 tx=0x7f7270004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd44fcee0361b621 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7257fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7257fff640 1 -- 172.21.15.37:0/907738512 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270012de0 con 0x55a05c5fb800 2024-04-23T18:32:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7257fff640 1 -- 172.21.15.37:0/907738512 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8623+0+0 (secure 0 0 0) 0x7f7270004b20 con 0x55a05c5fb800 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726c0316a0 con 0x55a05c5fb800 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7257fff640 1 -- 172.21.15.37:0/907738512 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7270004d20 con 0x55a05c5fb800 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7281783740 10 client.16785 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a05c60a320 con 0x55a05c5fb800 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.809+0000 7f7257fff640 1 -- 172.21.15.37:0/907738512 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c0316a0 con 0x55a05c5fb800 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7257fff640 1 -- 172.21.15.37:0/907738512 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f727001f450 con 0x55a05c5fb800 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7281783740 10 client.16785 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7281783740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7281783740 20 client.16785 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7281783740 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c60b830 0x55a05c60dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7281783740 4 client.16785 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05c563390 con 0x55a05c60b830 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f727c80e640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c60b830 0x55a05c60dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f7257fff640 10 client.16785 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.810+0000 7f727c80e640 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c60b830 0x55a05c60dc80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68095611d9bf8884 server_cookie=de2bb6fa9748c9f0 in_seq=0 out_seq=0 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7257fff640 1 -- 172.21.15.37:0/907738512 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a05c563390 con 0x55a05c60b830 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7257fff640 10 client.16785 handle_command_reply: tid=0 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 client.16785 shutdown 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 msgr2=0x55a05c5f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c5f8900 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f727000fa80 tx=0x7f7270004a40 comp rx=0 tx=0).stop 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 shutdown_connections 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c60b830 0x55a05c60dc80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c5f8900 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c5f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 --2- 172.21.15.37:0/907738512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c5fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7257fff640 0 client.16785 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 >> 172.21.15.37:0/907738512 conn(0x55a05c5ecb30 msgr2=0x55a05c5f28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 shutdown_connections 2024-04-23T18:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 1 -- 172.21.15.37:0/907738512 wait complete. 2024-04-23T18:32:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.811+0000 7f7281783740 20 client.16785 trim_cache size 0 max 16384 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.826+0000 7f7281783740 1 Processor -- start 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.826+0000 7f7281783740 1 -- start start 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7281783740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0620 0x55a05c5fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7281783740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0f40 0x55a05c613590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7281783740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c613ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7281783740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f72781b04a0 con 0x55a05c5fb800 2024-04-23T18:32:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7281783740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72781add40 con 0x55a05c5f0f40 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7281783740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72781abbb0 con 0x55a05c5f0620 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c613ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c613ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51468/0 (socket says 172.21.15.37:51468) 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 -- 172.21.15.37:0/569810628 learned_addr learned my addr 172.21.15.37:0/569810628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727c80e640 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0620 0x55a05c5fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 -- 172.21.15.37:0/569810628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0620 msgr2=0x55a05c5fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0620 0x55a05c5fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 -- 172.21.15.37:0/569810628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0f40 msgr2=0x55a05c613590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f7277fff640 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0f40 0x55a05c613590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0f40 0x55a05c613590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 -- 172.21.15.37:0/569810628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05c5f6950 con 0x55a05c5fb800 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727c80e640 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0620 0x55a05c5fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f727d00f640 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c613ad0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f727000fa50 tx=0x7f7270014f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e803e12532348 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.827+0000 7f72627fc640 1 -- 172.21.15.37:0/569810628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727003d020 con 0x55a05c5fb800 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f72627fc640 1 -- 172.21.15.37:0/569810628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7270003e20 con 0x55a05c5fb800 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f72627fc640 1 -- 172.21.15.37:0/569810628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270035040 con 0x55a05c5fb800 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 -- 172.21.15.37:0/569810628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 msgr2=0x55a05c613ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c613ad0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f727000fa50 tx=0x7f7270014f00 comp rx=0 tx=0).stop 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 -- 172.21.15.37:0/569810628 shutdown_connections 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c613ad0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0f40 0x55a05c613590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 --2- 172.21.15.37:0/569810628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0620 0x55a05c5fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 -- 172.21.15.37:0/569810628 >> 172.21.15.37:0/569810628 conn(0x55a05c5ecb30 msgr2=0x55a05c5f8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 -- 172.21.15.37:0/569810628 shutdown_connections 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 -- 172.21.15.37:0/569810628 wait complete. 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.828+0000 7f7281783740 1 -- start start 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 Processor -- start 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c603980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c600d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c601280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f72781ac200 con 0x55a05c5fb800 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72781ad670 con 0x55a05c5f0620 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72781ac8d0 con 0x55a05c5f0f40 2024-04-23T18:32:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7281783740 10 client.0 _ll_register_callbacks cb 0x7f72700097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f727d00f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c601280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c600d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c600d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:37634/0 (socket says 172.21.15.37:37634) 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 -- 172.21.15.37:0/3396502477 learned_addr learned my addr 172.21.15.37:0/3396502477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f727d00f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c601280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51476/0 (socket says 172.21.15.37:51476) 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f727c80e640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c603980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 -- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 msgr2=0x55a05c603980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c603980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 -- 172.21.15.37:0/3396502477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 msgr2=0x55a05c601280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c601280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f7277fff640 1 -- 172.21.15.37:0/3396502477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05c6018c0 con 0x55a05c5f0f40 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.829+0000 7f727d00f640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c601280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7277fff640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c600d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7264010bb0 tx=0x7f726406bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6e2b3515f55045 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7257fff640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7257fff640 1 -- 172.21.15.37:0/3396502477 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264066ed0 con 0x55a05c5f0f40 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7270003e20 con 0x55a05c5f0f40 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7257fff640 1 -- 172.21.15.37:0/3396502477 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8623+0+0 (secure 0 0 0) 0x7f726409b810 con 0x55a05c5f0f40 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7257fff640 1 -- 172.21.15.37:0/3396502477 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f72640b0e60 con 0x55a05c5f0f40 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 10 client.26203 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a05c671bf0 con 0x55a05c5f0f40 2024-04-23T18:32:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7257fff640 1 -- 172.21.15.37:0/3396502477 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7270003e20 con 0x55a05c5f0f40 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7257fff640 1 -- 172.21.15.37:0/3396502477 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f726409b280 con 0x55a05c5f0f40 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 10 client.26203 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 10 client.26203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 20 client.26203 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.830+0000 7f7281783740 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c672350 0x55a05c674770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.831+0000 7f7281783740 4 client.26203 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.831+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a05c55d200 con 0x55a05c672350 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.831+0000 7f727c80e640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c672350 0x55a05c674770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.831+0000 7f7257fff640 10 client.26203 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.831+0000 7f727c80e640 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c672350 0x55a05c674770 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca7b156f83a1d60 server_cookie=aae1b6bee5c54caf in_seq=0 out_seq=0 2024-04-23T18:32:06.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 570036 3.66 MB/sec execute 491 sec latency 2199.951 ms 2024-04-23T18:32:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7257fff640 1 -- 172.21.15.37:0/3396502477 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a05c55d200 con 0x55a05c672350 2024-04-23T18:32:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7257fff640 10 client.26203 handle_command_reply: tid=0 2024-04-23T18:32:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 client.26203 shutdown 2024-04-23T18:32:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 msgr2=0x55a05c600d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c600d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7264010bb0 tx=0x7f726406bca0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 shutdown_connections 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a05c672350 0x55a05c674770 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a05c5fb800 0x55a05c601280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a05c5f0f40 0x55a05c600d40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 --2- 172.21.15.37:0/3396502477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a05c5f0620 0x55a05c603980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7257fff640 0 client.26203 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.874+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 >> 172.21.15.37:0/3396502477 conn(0x55a05c5ecb30 msgr2=0x55a05c5eb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 shutdown_connections 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f7281783740 1 -- 172.21.15.37:0/3396502477 wait complete. 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f7281783740 20 client.26203 trim_cache size 0 max 16384 2024-04-23T18:32:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:32:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:32:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:32:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:32:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f72680a3dc0 msgr2=0x7f72680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f72680a3dc0 0x7f72680a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7270009370 tx=0x7f7270009570 comp rx=0 tx=0).stop 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 msgr2=0x7f7278198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 0x7f7278198bc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f726400d770 tx=0x7f726400dc30 comp rx=0 tx=0).stop 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 shutdown_connections 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f72680a3dc0 0x7f72680a6280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f727819cf40 0x7f727819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72781061b0 0x7f7278198bc0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 --2- 172.21.15.37:0/2428672986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72781010e0 0x7f7278198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 >> 172.21.15.37:0/2428672986 conn(0x7f72780771d0 msgr2=0x7f727806aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.875+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 shutdown_connections 2024-04-23T18:32:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:06.876+0000 7f727ea72640 1 -- 172.21.15.37:0/2428672986 wait complete. 2024-04-23T18:32:07.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 572212 3.68 MB/sec execute 492 sec latency 2200.198 ms 2024-04-23T18:32:08.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:08 smithi172 ceph-mon[32456]: pgmap v365: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:08.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:08 smithi173 ceph-mon[32906]: pgmap v365: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:08.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:08 smithi037 ceph-mon[29404]: pgmap v365: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:08.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 573410 3.68 MB/sec execute 493 sec latency 173.729 ms 2024-04-23T18:32:09.851 INFO:tasks.workunit.client.0.smithi037.stdout: 1 573579 3.68 MB/sec execute 494 sec latency 911.027 ms 2024-04-23T18:32:10.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:10 smithi172 ceph-mon[32456]: pgmap v366: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:10.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:10 smithi173 ceph-mon[32906]: pgmap v366: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:10.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:10 smithi037 ceph-mon[29404]: pgmap v366: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:10.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 573579 3.67 MB/sec execute 495 sec latency 1911.164 ms 2024-04-23T18:32:11.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 573579 3.66 MB/sec execute 496 sec latency 2911.203 ms 2024-04-23T18:32:12.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:12 smithi172 ceph-mon[32456]: pgmap v367: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:12.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:12 smithi172 ceph-mon[32456]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:32:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:12 smithi173 ceph-mon[32906]: pgmap v367: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:12 smithi173 ceph-mon[32906]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:32:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:12 smithi037 ceph-mon[29404]: pgmap v367: 97 pgs: 97 active+clean; 415 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-04-23T18:32:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:12 smithi037 ceph-mon[29404]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:32:12.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 574318 3.66 MB/sec execute 497 sec latency 3641.888 ms 2024-04-23T18:32:13.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 576027 3.68 MB/sec execute 498 sec latency 14.707 ms 2024-04-23T18:32:14.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:14 smithi172 ceph-mon[32456]: pgmap v369: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-04-23T18:32:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:14 smithi173 ceph-mon[32906]: pgmap v369: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-04-23T18:32:14.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:14 smithi037 ceph-mon[29404]: pgmap v369: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-04-23T18:32:14.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 577126 3.68 MB/sec execute 499 sec latency 391.196 ms 2024-04-23T18:32:15.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 577126 3.67 MB/sec execute 500 sec latency 1391.317 ms 2024-04-23T18:32:16.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:16 smithi172 ceph-mon[32456]: pgmap v370: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-23T18:32:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:16 smithi173 ceph-mon[32906]: pgmap v370: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-23T18:32:16.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:16 smithi037 ceph-mon[29404]: pgmap v370: 97 pgs: 97 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-23T18:32:16.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 577126 3.66 MB/sec execute 501 sec latency 2391.414 ms 2024-04-23T18:32:17.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 577126 3.66 MB/sec execute 502 sec latency 3391.522 ms 2024-04-23T18:32:18.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:18 smithi172 ceph-mon[32456]: pgmap v371: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:18 smithi173 ceph-mon[32906]: pgmap v371: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:18.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:18 smithi037 ceph-mon[29404]: pgmap v371: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:18.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 577890 3.65 MB/sec execute 503 sec latency 4101.011 ms 2024-04-23T18:32:19.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 579666 3.67 MB/sec execute 504 sec latency 15.605 ms 2024-04-23T18:32:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 -- 172.21.15.37:0/2899277956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe41021c0 msgr2=0x7f4fe41025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 --2- 172.21.15.37:0/2899277956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe41021c0 0x7f4fe41025a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009900 tx=0x7f4fd802a410 comp rx=0 tx=0).stop 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 -- 172.21.15.37:0/2899277956 shutdown_connections 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 --2- 172.21.15.37:0/2899277956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fe4107290 0x7f4fe410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 --2- 172.21.15.37:0/2899277956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe4102b70 0x7f4fe4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 --2- 172.21.15.37:0/2899277956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe41021c0 0x7f4fe41025a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 -- 172.21.15.37:0/2899277956 >> 172.21.15.37:0/2899277956 conn(0x7f4fe40fda20 msgr2=0x7f4fe40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 -- 172.21.15.37:0/2899277956 shutdown_connections 2024-04-23T18:32:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.970+0000 7f4feab8e640 1 -- 172.21.15.37:0/2899277956 wait complete. 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 Processor -- start 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 -- start start 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 0x7f4fe4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fe4107290 0x7f4fe4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe4074cd0 0x7f4fe41a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41121d0 con 0x7f4fe4074cd0 2024-04-23T18:32:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4112050 con 0x7f4fe4107290 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4feab8e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4112350 con 0x7f4fe41021c0 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 0x7f4fe4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe912b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe4074cd0 0x7f4fe41a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 0x7f4fe4074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:37416/0 (socket says 172.21.15.37:37416) 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 -- 172.21.15.37:0/1642135277 learned_addr learned my addr 172.21.15.37:0/1642135277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fe4107290 0x7f4fe4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 -- 172.21.15.37:0/1642135277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fe4107290 msgr2=0x7f4fe4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fe4107290 0x7f4fe4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 -- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe4074cd0 msgr2=0x7f4fe41a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.971+0000 7f4fe892a640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe4074cd0 0x7f4fe41a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4fe892a640 1 -- 172.21.15.37:0/1642135277 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a6fe0 con 0x7f4fe41021c0 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4fe912b640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe4074cd0 0x7f4fe41a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4fe892a640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 0x7f4fe4074250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8005940 tx=0x7f4fd80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4fe1ffb640 1 -- 172.21.15.37:0/1642135277 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd803f030 con 0x7f4fe41021c0 2024-04-23T18:32:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a7270 con 0x7f4fe41021c0 2024-04-23T18:32:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4fe1ffb640 1 -- 172.21.15.37:0/1642135277 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd80044c0 con 0x7f4fe41021c0 2024-04-23T18:32:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4fe1ffb640 1 -- 172.21.15.37:0/1642135277 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8035070 con 0x7f4fe41021c0 2024-04-23T18:32:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.972+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41a7760 con 0x7f4fe41021c0 2024-04-23T18:32:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.974+0000 7f4fed97d740 1 Processor -- start 2024-04-23T18:32:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.974+0000 7f4fe1ffb640 1 -- 172.21.15.37:0/1642135277 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fd8042030 con 0x7f4fe41021c0 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 -- start start 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226de6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226df12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226df3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564226d4b970 con 0x564226de6f40 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564226ce84a0 con 0x564226de6620 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fed97d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564226d47930 con 0x564226df1800 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226df12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe912b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226de6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226df12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54942/0 (socket says 172.21.15.37:54942) 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 -- 172.21.15.37:0/2087071606 learned_addr learned my addr 172.21.15.37:0/2087071606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226df3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 -- 172.21.15.37:0/2087071606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 msgr2=0x564226df3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226df3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 -- 172.21.15.37:0/2087071606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 msgr2=0x564226de6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226de6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 -- 172.21.15.37:0/2087071606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564226a5e210 con 0x564226de6f40 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe912b640 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226de6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe1ffb640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4fd40a3dc0 0x7f4fd40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.975+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226df12c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80ee1d0 tx=0x7f4fd80f4e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6aabb1eab58f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fe912b640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4fd40a3dc0 0x7f4fd40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fceffd640 1 -- 172.21.15.37:0/2087071606 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80b8e30 con 0x564226de6f40 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fe1ffb640 1 -- 172.21.15.37:0/1642135277 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f4fd80f0be0 con 0x7f4fe41021c0 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fceffd640 1 -- 172.21.15.37:0/2087071606 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd80c0e90 con 0x564226de6f40 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fceffd640 1 -- 172.21.15.37:0/2087071606 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80cada0 con 0x564226de6f40 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fe912b640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4fd40a3dc0 0x7f4fd40a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009fd0 tx=0x7f4fdc009500 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 -- 172.21.15.37:0/2087071606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 msgr2=0x564226df12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226df12c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80ee1d0 tx=0x7f4fd80f4e10 comp rx=0 tx=0).stop 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 -- 172.21.15.37:0/2087071606 shutdown_connections 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226df3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226df12c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2087071606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226de6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.976+0000 7f4fed97d740 1 -- 172.21.15.37:0/2087071606 >> 172.21.15.37:0/2087071606 conn(0x564226de2b30 msgr2=0x564226de2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 -- 172.21.15.37:0/2087071606 shutdown_connections 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 -- 172.21.15.37:0/2087071606 wait complete. 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 -- start start 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 Processor -- start 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226df1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226dee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226dee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564226d4b970 con 0x564226de6620 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564226ce60b0 con 0x564226de6f40 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564226df41f0 con 0x564226df1800 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fed97d740 10 client.0 _ll_register_callbacks cb 0x564226cd3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226df1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226df1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54944/0 (socket says 172.21.15.37:54944) 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fe892a640 1 -- 172.21.15.37:0/2113289049 learned_addr learned my addr 172.21.15.37:0/2113289049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe912b640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226dee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.977+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226dee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe892a640 1 -- 172.21.15.37:0/2113289049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 msgr2=0x564226dee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226dee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe892a640 1 -- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 msgr2=0x564226dee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226dee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe892a640 1 -- 172.21.15.37:0/2113289049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564226b2a620 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226df1100 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80cb160 tx=0x7f4fd8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ca6d79fa8ad03b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fc3fff640 1 -- 172.21.15.37:0/2113289049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8049030 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fc3fff640 1 -- 172.21.15.37:0/2113289049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f4fd8003cf0 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd80c0e90 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fc3fff640 1 -- 172.21.15.37:0/2113289049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4fd80cafa0 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fed97d740 10 client.16803 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564226dfff40 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.978+0000 7f4fc3fff640 1 -- 172.21.15.37:0/2113289049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd80c0e90 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fc3fff640 1 -- 172.21.15.37:0/2113289049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f4fd80730f0 con 0x564226de6620 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fed97d740 10 client.16803 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fed97d740 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fed97d740 20 client.16803 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e012e0 0x564226e03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fed97d740 4 client.16803 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564226d59390 con 0x564226e012e0 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e012e0 0x564226e03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fc3fff640 10 client.16803 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.979+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e012e0 0x564226e03750 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff6482944218eb7 server_cookie=46a26dd009267d2d in_seq=0 out_seq=0 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fc3fff640 1 -- 172.21.15.37:0/2113289049 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564226d59390 con 0x564226e012e0 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fc3fff640 10 client.16803 handle_command_reply: tid=0 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 client.16803 shutdown 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 msgr2=0x564226df1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226df1100 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80cb160 tx=0x7f4fd8005c90 comp rx=0 tx=0).stop 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 shutdown_connections 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e012e0 0x564226e03750 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226dee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226dee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2113289049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226df1100 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fc3fff640 0 client.16803 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.980+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 >> 172.21.15.37:0/2113289049 conn(0x564226de2b30 msgr2=0x564226de88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.981+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 shutdown_connections 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.981+0000 7f4fed97d740 1 -- 172.21.15.37:0/2113289049 wait complete. 2024-04-23T18:32:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.981+0000 7f4fed97d740 20 client.16803 trim_cache size 0 max 16384 2024-04-23T18:32:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 Processor -- start 2024-04-23T18:32:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 -- start start 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226dfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226df0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226e09020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b2670 con 0x564226de6620 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41aff10 con 0x564226de6f40 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fed97d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41add80 con 0x564226df1800 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.996+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226dfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226dfcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54958/0 (socket says 172.21.15.37:54958) 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 -- 172.21.15.37:0/2683645283 learned_addr learned my addr 172.21.15.37:0/2683645283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226df0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe912b640 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226e09020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 -- 172.21.15.37:0/2683645283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 msgr2=0x564226e09020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226e09020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 -- 172.21.15.37:0/2683645283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 msgr2=0x564226df0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226df0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 -- 172.21.15.37:0/2683645283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564226de6d70 con 0x564226de6620 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe912b640 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226e09020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fe892a640 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226dfcf30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006010 tx=0x7f4fd803b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ee3281c5035b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fce7fc640 1 -- 172.21.15.37:0/2683645283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8049070 con 0x564226de6620 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fce7fc640 1 -- 172.21.15.37:0/2683645283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd80f4140 con 0x564226de6620 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.997+0000 7f4fce7fc640 1 -- 172.21.15.37:0/2683645283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80cb190 con 0x564226de6620 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 -- 172.21.15.37:0/2683645283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 msgr2=0x564226dfcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226dfcf30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006010 tx=0x7f4fd803b4f0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 -- 172.21.15.37:0/2683645283 shutdown_connections 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226e09020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6f40 0x564226df0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 --2- 172.21.15.37:0/2683645283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6620 0x564226dfcf30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 -- 172.21.15.37:0/2683645283 >> 172.21.15.37:0/2683645283 conn(0x564226de2b30 msgr2=0x564226dee510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 -- 172.21.15.37:0/2683645283 shutdown_connections 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 -- 172.21.15.37:0/2683645283 wait complete. 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 -- start start 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.998+0000 7f4fed97d740 1 Processor -- start 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226df9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226deace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226deb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41ae3d0 con 0x564226de6f40 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41af840 con 0x564226de6620 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41aeaa0 con 0x564226df1800 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 10 client.0 _ll_register_callbacks cb 0x7f4fd80f4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226deace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226deace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54972/0 (socket says 172.21.15.37:54972) 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 -- 172.21.15.37:0/444927444 learned_addr learned my addr 172.21.15.37:0/444927444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe892a640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226df9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe912b640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226deb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 -- 172.21.15.37:0/444927444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 msgr2=0x564226deb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226deb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 -- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 msgr2=0x564226df9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226df9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 -- 172.21.15.37:0/444927444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564226deb8e0 con 0x564226de6f40 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fe3fff640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226deace0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4075c90 tx=0x7f4fd0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a579488c65315364 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fc3fff640 1 -- 172.21.15.37:0/444927444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0027610 con 0x564226de6f40 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fc3fff640 1 -- 172.21.15.37:0/444927444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f4fd001e400 con 0x564226de6f40 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:19.999+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd80f4140 con 0x564226de6f40 2024-04-23T18:32:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fc3fff640 1 -- 172.21.15.37:0/444927444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4fd00277b0 con 0x564226de6f40 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 10 client.16809 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564226e67740 con 0x564226de6f40 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fc3fff640 1 -- 172.21.15.37:0/444927444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd80f4140 con 0x564226de6f40 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fc3fff640 1 -- 172.21.15.37:0/444927444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f4fd001eb10 con 0x564226de6f40 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 10 client.16809 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 20 client.16809 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e67e40 0x564226e6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 4 client.16809 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564226d53200 con 0x564226e67e40 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.000+0000 7f4fe892a640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e67e40 0x564226e6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.001+0000 7f4fe892a640 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e67e40 0x564226e6a260 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6485a80f73d9894 server_cookie=37f55cf93b814952 in_seq=0 out_seq=0 2024-04-23T18:32:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.001+0000 7f4fc3fff640 10 client.16809 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.033+0000 7f4fc3fff640 1 -- 172.21.15.37:0/444927444 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564226d53200 con 0x564226e67e40 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fc3fff640 10 client.16809 handle_command_reply: tid=0 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 client.16809 shutdown 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 msgr2=0x564226deace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226deace0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4075c90 tx=0x7f4fd0010800 comp rx=0 tx=0).stop 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 shutdown_connections 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564226e67e40 0x564226e6a260 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564226df1800 0x564226deb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564226de6f40 0x564226deace0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 --2- 172.21.15.37:0/444927444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564226de6620 0x564226df9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 >> 172.21.15.37:0/444927444 conn(0x564226de2b30 msgr2=0x564226de7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fc3fff640 0 client.16809 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 shutdown_connections 2024-04-23T18:32:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 1 -- 172.21.15.37:0/444927444 wait complete. 2024-04-23T18:32:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.034+0000 7f4fed97d740 20 client.16809 trim_cache size 0 max 16384 2024-04-23T18:32:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:32:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:32:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:32:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4fd40a3dc0 msgr2=0x7f4fd40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4fd40a3dc0 0x7f4fd40a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009fd0 tx=0x7f4fdc009500 comp rx=0 tx=0).stop 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 msgr2=0x7f4fe4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 0x7f4fe4074250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8005940 tx=0x7f4fd80040c0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 shutdown_connections 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f4fd40a3dc0 0x7f4fd40a6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fe4074cd0 0x7f4fe41a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fe4107290 0x7f4fe4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 --2- 172.21.15.37:0/1642135277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fe41021c0 0x7f4fe4074250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 >> 172.21.15.37:0/1642135277 conn(0x7f4fe40fda20 msgr2=0x7f4fe40ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 shutdown_connections 2024-04-23T18:32:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:20.035+0000 7f4feab8e640 1 -- 172.21.15.37:0/1642135277 wait complete. 2024-04-23T18:32:20.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:20 smithi172 ceph-mon[32456]: pgmap v372: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:20.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:20 smithi173 ceph-mon[32906]: pgmap v372: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:20.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:20 smithi037 ceph-mon[29404]: pgmap v372: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:20.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 580669 3.67 MB/sec execute 505 sec latency 396.157 ms 2024-04-23T18:32:21.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 580669 3.67 MB/sec execute 506 sec latency 1396.193 ms 2024-04-23T18:32:22.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:22 smithi172 ceph-mon[32456]: pgmap v373: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:22 smithi173 ceph-mon[32906]: pgmap v373: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:22.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:22 smithi037 ceph-mon[29404]: pgmap v373: 97 pgs: 97 active+clean; 422 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.0 MiB/s wr, 138 op/s 2024-04-23T18:32:22.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 581406 3.66 MB/sec execute 507 sec latency 2123.350 ms 2024-04-23T18:32:23.852 INFO:tasks.workunit.client.0.smithi037.stdout: 1 583114 3.68 MB/sec execute 508 sec latency 13.553 ms 2024-04-23T18:32:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:24 smithi172 ceph-mon[32456]: pgmap v374: 97 pgs: 97 active+clean; 426 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-04-23T18:32:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:24 smithi173 ceph-mon[32906]: pgmap v374: 97 pgs: 97 active+clean; 426 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-04-23T18:32:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:24 smithi037 ceph-mon[29404]: pgmap v374: 97 pgs: 97 active+clean; 426 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-04-23T18:32:24.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 584222 3.68 MB/sec execute 509 sec latency 342.585 ms 2024-04-23T18:32:25.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 584222 3.67 MB/sec execute 510 sec latency 1342.680 ms 2024-04-23T18:32:26.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:26 smithi172 ceph-mon[32456]: pgmap v375: 97 pgs: 97 active+clean; 426 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-04-23T18:32:26.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:26 smithi173 ceph-mon[32906]: pgmap v375: 97 pgs: 97 active+clean; 426 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-04-23T18:32:26.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:26 smithi037 ceph-mon[29404]: pgmap v375: 97 pgs: 97 active+clean; 426 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.7 MiB/s wr, 119 op/s 2024-04-23T18:32:26.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 584222 3.67 MB/sec execute 511 sec latency 2342.844 ms 2024-04-23T18:32:27.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 584222 3.66 MB/sec execute 512 sec latency 3342.969 ms 2024-04-23T18:32:28.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:28 smithi172 ceph-mon[32456]: pgmap v376: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-04-23T18:32:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:28 smithi173 ceph-mon[32906]: pgmap v376: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-04-23T18:32:28.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:28 smithi037 ceph-mon[29404]: pgmap v376: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-04-23T18:32:28.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 584985 3.66 MB/sec execute 513 sec latency 4047.509 ms 2024-04-23T18:32:29.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 586690 3.67 MB/sec execute 514 sec latency 17.042 ms 2024-04-23T18:32:30.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:30 smithi172 ceph-mon[32456]: pgmap v377: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-23T18:32:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:30 smithi173 ceph-mon[32906]: pgmap v377: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-23T18:32:30.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:30 smithi037 ceph-mon[29404]: pgmap v377: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-23T18:32:30.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 587763 3.68 MB/sec execute 515 sec latency 418.537 ms 2024-04-23T18:32:31.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 587763 3.67 MB/sec execute 516 sec latency 1418.636 ms 2024-04-23T18:32:32.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:32 smithi172 ceph-mon[32456]: pgmap v378: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-23T18:32:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:32 smithi173 ceph-mon[32906]: pgmap v378: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-23T18:32:32.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:32 smithi037 ceph-mon[29404]: pgmap v378: 97 pgs: 97 active+clean; 433 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-04-23T18:32:32.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 588509 3.67 MB/sec execute 517 sec latency 2152.445 ms 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/2912096239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78101a90 msgr2=0x7f9f78105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/2912096239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78101a90 0x7f9f78105ad0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009900 tx=0x7f9f6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/2912096239 shutdown_connections 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/2912096239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781061b0 0x7f9f7810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/2912096239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78101a90 0x7f9f78105ad0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/2912096239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f781010e0 0x7f9f781014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/2912096239 >> 172.21.15.37:0/2912096239 conn(0x7f9f780771d0 msgr2=0x7f9f780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/2912096239 shutdown_connections 2024-04-23T18:32:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/2912096239 wait complete. 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 Processor -- start 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.127+0000 7f9f7ec9e640 1 -- start start 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781010e0 0x7f9f78198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f781061b0 0x7f9f78198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 0x7f9f7819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781110d0 con 0x7f9f7819cfc0 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78110f50 con 0x7f9f781061b0 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78111250 con 0x7f9f781010e0 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 0x7f9f7819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781010e0 0x7f9f78198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f781061b0 0x7f9f78198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781010e0 0x7f9f78198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:40054/0 (socket says 172.21.15.37:40054) 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 0x7f9f7819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60674/0 (socket says 172.21.15.37:60674) 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/1954463955 learned_addr learned my addr 172.21.15.37:0/1954463955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 -- 172.21.15.37:0/1954463955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781010e0 msgr2=0x7f9f78198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781010e0 0x7f9f78198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 -- 172.21.15.37:0/1954463955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f781061b0 msgr2=0x7f9f78198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f781061b0 0x7f9f78198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 -- 172.21.15.37:0/1954463955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7819dab0 con 0x7f9f7819cfc0 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 0x7f9f7819d470 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e940 tx=0x7f9f7000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f75ffb640 1 -- 172.21.15.37:0/1954463955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70004d20 con 0x7f9f7819cfc0 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.128+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7819dda0 con 0x7f9f7819cfc0 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.129+0000 7f9f75ffb640 1 -- 172.21.15.37:0/1954463955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f70005020 con 0x7f9f7819cfc0 2024-04-23T18:32:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.129+0000 7f9f75ffb640 1 -- 172.21.15.37:0/1954463955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7001f060 con 0x7f9f7819cfc0 2024-04-23T18:32:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.129+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f781a5560 con 0x7f9f7819cfc0 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.131+0000 7f9f75ffb640 1 -- 172.21.15.37:0/1954463955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f70005460 con 0x7f9f7819cfc0 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.131+0000 7f9f819b0740 1 Processor -- start 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.131+0000 7f9f819b0740 1 -- start start 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f75ffb640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9f600a3dc0 0x7f9f600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33e20 0x5653d3d426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d45000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f819b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5653d3c98970 con 0x5653d3d33e20 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f819b0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653d3c354a0 con 0x5653d3d33500 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f819b0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5653d3c94930 con 0x5653d3d42c10 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f77fff640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9f600a3dc0 0x7f9f600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:54062/0 (socket says 172.21.15.37:54062) 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7d23b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d45000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/1460414243 learned_addr learned my addr 172.21.15.37:0/1460414243 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f75ffb640 1 -- 172.21.15.37:0/1954463955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f9f700d0820 con 0x7f9f7819cfc0 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f77fff640 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33e20 0x5653d3d426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/1460414243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 msgr2=0x5653d3d45000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d45000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/1460414243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33e20 msgr2=0x5653d3d426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33e20 0x5653d3d426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/1460414243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d39ab210 con 0x5653d3d33500 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f77fff640 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33e20 0x5653d3d426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f77fff640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9f600a3dc0 0x7f9f600a6280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c000c00 tx=0x7f9f6c005780 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.132+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d338e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009e40 tx=0x7f9f68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5a88a2c41a4d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f5effd640 1 -- 172.21.15.37:0/1460414243 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6801e070 con 0x5653d3d33500 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f5effd640 1 -- 172.21.15.37:0/1460414243 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f680198c0 con 0x5653d3d33500 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f5effd640 1 -- 172.21.15.37:0/1460414243 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800f8f0 con 0x5653d3d33500 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 -- 172.21.15.37:0/1460414243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 msgr2=0x5653d3d338e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d338e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009e40 tx=0x7f9f68011810 comp rx=0 tx=0).stop 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 -- 172.21.15.37:0/1460414243 shutdown_connections 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d45000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33e20 0x5653d3d426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 --2- 172.21.15.37:0/1460414243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d338e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.133+0000 7f9f819b0740 1 -- 172.21.15.37:0/1460414243 >> 172.21.15.37:0/1460414243 conn(0x5653d3d2fb30 msgr2=0x5653d3d2ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 -- 172.21.15.37:0/1460414243 shutdown_connections 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 -- 172.21.15.37:0/1460414243 wait complete. 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 -- start start 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 Processor -- start 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d33e20 0x5653d3d3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d42c10 0x5653d3d3fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5653d3d45840 con 0x5653d3d42c10 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653d3c330b0 con 0x5653d3d33500 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5653d3d459c0 con 0x5653d3d33e20 2024-04-23T18:32:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f819b0740 10 client.0 _ll_register_callbacks cb 0x5653d3c20d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f77fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d33e20 0x5653d3d3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.134+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d42520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:54066/0 (socket says 172.21.15.37:54066) 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/234118468 learned_addr learned my addr 172.21.15.37:0/234118468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d42c10 0x5653d3d3fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 -- 172.21.15.37:0/234118468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d33e20 msgr2=0x5653d3d3f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d33e20 0x5653d3d3f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 -- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 msgr2=0x5653d3d42520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d42520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 -- 172.21.15.37:0/234118468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d3a77620 con 0x5653d3d42c10 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d42520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f77fff640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d33e20 0x5653d3d3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d42c10 0x5653d3d3fd50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9f700cdf30 tx=0x7f9f7009c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276650efbefe9f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f57fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f57fff640 1 -- 172.21.15.37:0/234118468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700a0b30 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f680198c0 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f57fff640 1 -- 172.21.15.37:0/234118468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f9f700870c0 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f57fff640 1 -- 172.21.15.37:0/234118468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9f700a53f0 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f819b0740 10 client.16821 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.135+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5653d3d51800 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f57fff640 1 -- 172.21.15.37:0/234118468 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f680198c0 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f57fff640 1 -- 172.21.15.37:0/234118468 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f9f7000f040 con 0x5653d3d42c10 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f819b0740 10 client.16821 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f819b0740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f819b0740 20 client.16821 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f819b0740 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3d55100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f819b0740 4 client.16821 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d3ca6390 con 0x5653d3d52d10 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3d55100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f57fff640 10 client.16821 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.136+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3d55100 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ad5aa7198159d0 server_cookie=dacdf067218fe559 in_seq=0 out_seq=0 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f57fff640 1 -- 172.21.15.37:0/234118468 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653d3ca6390 con 0x5653d3d52d10 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f57fff640 10 client.16821 handle_command_reply: tid=0 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 client.16821 shutdown 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d42c10 msgr2=0x5653d3d3fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d42c10 0x5653d3d3fd50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9f700cdf30 tx=0x7f9f7009c190 comp rx=0 tx=0).stop 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 shutdown_connections 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3d55100 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d42c10 0x5653d3d3fd50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d33e20 0x5653d3d3f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f57fff640 0 client.16821 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 --2- 172.21.15.37:0/234118468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33500 0x5653d3d42520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 >> 172.21.15.37:0/234118468 conn(0x5653d3d2fb30 msgr2=0x5653d3d359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 shutdown_connections 2024-04-23T18:32:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 1 -- 172.21.15.37:0/234118468 wait complete. 2024-04-23T18:32:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.137+0000 7f9f819b0740 20 client.16821 trim_cache size 0 max 16384 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 Processor -- start 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 -- start start 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d42050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781ae2c0 con 0x5653d3d33500 2024-04-23T18:32:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781abb60 con 0x5653d3d33e20 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f819b0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781a99d0 con 0x5653d3d42c10 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60698/0 (socket says 172.21.15.37:60698) 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.155+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4104519092 learned_addr learned my addr 172.21.15.37:0/4104519092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f77fff640 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d42050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4104519092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 msgr2=0x5653d3d5aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d5aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4104519092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 msgr2=0x5653d3d42050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d42050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4104519092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d3d2f960 con 0x5653d3d33500 2024-04-23T18:32:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4e740 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009bd0 tx=0x7f9f6800f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edf89272fcef85e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f5e7fc640 1 -- 172.21.15.37:0/4104519092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6802f070 con 0x5653d3d33500 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f5e7fc640 1 -- 172.21.15.37:0/4104519092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f6802a760 con 0x5653d3d33500 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.156+0000 7f9f5e7fc640 1 -- 172.21.15.37:0/4104519092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6801e040 con 0x5653d3d33500 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 -- 172.21.15.37:0/4104519092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 msgr2=0x5653d3d4e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4e740 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009bd0 tx=0x7f9f6800f4d0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 -- 172.21.15.37:0/4104519092 shutdown_connections 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d42050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4104519092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4e740 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 -- 172.21.15.37:0/4104519092 >> 172.21.15.37:0/4104519092 conn(0x5653d3d2fb30 msgr2=0x5653d3d3f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 -- 172.21.15.37:0/4104519092 shutdown_connections 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 -- 172.21.15.37:0/4104519092 wait complete. 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 -- start start 2024-04-23T18:32:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.157+0000 7f9f819b0740 1 Processor -- start 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d3c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781aa020 con 0x5653d3d33500 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781ab490 con 0x5653d3d33e20 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781aa6f0 con 0x5653d3d42c10 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 10 client.0 _ll_register_callbacks cb 0x7f9f7009c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60702/0 (socket says 172.21.15.37:60702) 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4174501933 learned_addr learned my addr 172.21.15.37:0/4174501933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f77fff640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7d23b640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d3c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4174501933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 msgr2=0x5653d3d3c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d3c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 msgr2=0x5653d3d3bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d3bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 -- 172.21.15.37:0/4174501933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d3d45000 con 0x5653d3d33500 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f7ca3a640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4ae40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9f6802d040 tx=0x7f9f68005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b8e6dca5995f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f57fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f57fff640 1 -- 172.21.15.37:0/4174501933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68037060 con 0x5653d3d33500 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f57fff640 1 -- 172.21.15.37:0/4174501933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f9f6802a2b0 con 0x5653d3d33500 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.158+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6802a760 con 0x5653d3d33500 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f57fff640 1 -- 172.21.15.37:0/4174501933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9f6801e030 con 0x5653d3d33500 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 10 client.16827 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5653d3d2f550 con 0x5653d3d33500 2024-04-23T18:32:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f57fff640 1 -- 172.21.15.37:0/4174501933 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f6802a760 con 0x5653d3d33500 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f57fff640 1 -- 172.21.15.37:0/4174501933 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7f9f680035f0 con 0x5653d3d33500 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 10 client.16827 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 20 client.16827 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3dbb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 4 client.16827 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653d3ca0200 con 0x5653d3d52d10 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.159+0000 7f9f77fff640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3dbb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.160+0000 7f9f57fff640 10 client.16827 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.160+0000 7f9f77fff640 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3dbb150 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcc986d7b27ec92 server_cookie=8fda02fe24928c0 in_seq=0 out_seq=0 2024-04-23T18:32:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.193+0000 7f9f57fff640 1 -- 172.21.15.37:0/4174501933 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653d3ca0200 con 0x5653d3d52d10 2024-04-23T18:32:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.193+0000 7f9f57fff640 10 client.16827 handle_command_reply: tid=0 2024-04-23T18:32:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.193+0000 7f9f819b0740 1 client.16827 shutdown 2024-04-23T18:32:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.193+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 msgr2=0x5653d3d4ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.193+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4ae40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9f6802d040 tx=0x7f9f68005db0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 shutdown_connections 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x5653d3d52d10 0x5653d3dbb150 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5653d3d42c10 0x5653d3d3c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f57fff640 0 client.16827 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653d3d33e20 0x5653d3d3bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 --2- 172.21.15.37:0/4174501933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653d3d33500 0x5653d3d4ae40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 >> 172.21.15.37:0/4174501933 conn(0x5653d3d2fb30 msgr2=0x5653d3d3a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 shutdown_connections 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 1 -- 172.21.15.37:0/4174501933 wait complete. 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f819b0740 20 client.16827 trim_cache size 0 max 16384 2024-04-23T18:32:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:32:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:32:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:32:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:32:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9f600a3dc0 msgr2=0x7f9f600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9f600a3dc0 0x7f9f600a6280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c000c00 tx=0x7f9f6c005780 comp rx=0 tx=0).stop 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 msgr2=0x7f9f7819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.194+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 0x7f9f7819d470 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e940 tx=0x7f9f7000ee00 comp rx=0 tx=0).stop 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 shutdown_connections 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f9f600a3dc0 0x7f9f600a6280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f7819cfc0 0x7f9f7819d470 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f781061b0 0x7f9f78198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 --2- 172.21.15.37:0/1954463955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f781010e0 0x7f9f78198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 >> 172.21.15.37:0/1954463955 conn(0x7f9f780771d0 msgr2=0x7f9f78104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 shutdown_connections 2024-04-23T18:32:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:33.195+0000 7f9f7ec9e640 1 -- 172.21.15.37:0/1954463955 wait complete. 2024-04-23T18:32:33.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 590240 3.68 MB/sec execute 518 sec latency 15.903 ms 2024-04-23T18:32:34.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:34 smithi172 ceph-mon[32456]: pgmap v379: 97 pgs: 97 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-04-23T18:32:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:34 smithi173 ceph-mon[32906]: pgmap v379: 97 pgs: 97 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-04-23T18:32:34.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:34 smithi037 ceph-mon[29404]: pgmap v379: 97 pgs: 97 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-04-23T18:32:34.853 INFO:tasks.workunit.client.0.smithi037.stdout: 1 591312 3.68 MB/sec execute 519 sec latency 431.866 ms 2024-04-23T18:32:35.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 591312 3.68 MB/sec execute 520 sec latency 1431.969 ms 2024-04-23T18:32:36.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:36 smithi172 ceph-mon[32456]: pgmap v380: 97 pgs: 97 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-04-23T18:32:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:36 smithi173 ceph-mon[32906]: pgmap v380: 97 pgs: 97 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-04-23T18:32:36.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:36 smithi037 ceph-mon[29404]: pgmap v380: 97 pgs: 97 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-04-23T18:32:36.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 591312 3.67 MB/sec execute 521 sec latency 2432.075 ms 2024-04-23T18:32:37.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 591312 3.66 MB/sec execute 522 sec latency 3432.142 ms 2024-04-23T18:32:38.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:38 smithi172 ceph-mon[32456]: pgmap v381: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-04-23T18:32:38.357 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:38 smithi173 ceph-mon[32906]: pgmap v381: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-04-23T18:32:38.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:38 smithi037 ceph-mon[29404]: pgmap v381: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-04-23T18:32:38.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 592077 3.66 MB/sec execute 523 sec latency 4139.131 ms 2024-04-23T18:32:39.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 593841 3.68 MB/sec execute 524 sec latency 16.811 ms 2024-04-23T18:32:40.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:40 smithi172 ceph-mon[32456]: pgmap v382: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-23T18:32:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:40 smithi173 ceph-mon[32906]: pgmap v382: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-23T18:32:40.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:40 smithi037 ceph-mon[29404]: pgmap v382: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-23T18:32:40.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 594856 3.68 MB/sec execute 525 sec latency 450.477 ms 2024-04-23T18:32:41.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 594856 3.67 MB/sec execute 526 sec latency 1450.582 ms 2024-04-23T18:32:42.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:42 smithi172 ceph-mon[32456]: pgmap v383: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-23T18:32:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:42 smithi173 ceph-mon[32906]: pgmap v383: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-23T18:32:42.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:42 smithi037 ceph-mon[29404]: pgmap v383: 97 pgs: 97 active+clean; 442 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-04-23T18:32:42.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 595600 3.67 MB/sec execute 527 sec latency 2179.612 ms 2024-04-23T18:32:43.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 597276 3.68 MB/sec execute 528 sec latency 13.131 ms 2024-04-23T18:32:44.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:44 smithi172 ceph-mon[32456]: pgmap v384: 97 pgs: 97 active+clean; 447 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-04-23T18:32:44.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:44 smithi173 ceph-mon[32906]: pgmap v384: 97 pgs: 97 active+clean; 447 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-04-23T18:32:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:44 smithi037 ceph-mon[29404]: pgmap v384: 97 pgs: 97 active+clean; 447 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 157 op/s 2024-04-23T18:32:44.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 598403 3.69 MB/sec execute 529 sec latency 339.558 ms 2024-04-23T18:32:45.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 598403 3.68 MB/sec execute 530 sec latency 1339.671 ms 2024-04-23T18:32:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 -- 172.21.15.37:0/3533296632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd201031e0 msgr2=0x7fdd201035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 --2- 172.21.15.37:0/3533296632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd201031e0 0x7fdd201035c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd1002a950 comp rx=0 tx=0).stop 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 -- 172.21.15.37:0/3533296632 shutdown_connections 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 --2- 172.21.15.37:0/3533296632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd2010c5b0 0x7fdd2010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 --2- 172.21.15.37:0/3533296632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd20103b00 0x7fdd2010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 --2- 172.21.15.37:0/3533296632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd201031e0 0x7fdd201035c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 -- 172.21.15.37:0/3533296632 >> 172.21.15.37:0/3533296632 conn(0x7fdd20077fd0 msgr2=0x7fdd200783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 -- 172.21.15.37:0/3533296632 shutdown_connections 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.294+0000 7fdd25805640 1 -- 172.21.15.37:0/3533296632 wait complete. 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 Processor -- start 2024-04-23T18:32:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 -- start start 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd201031e0 0x7fdd20100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd20103b00 0x7fdd20101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 0x7fdd201018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20112200 con 0x7fdd2010c5b0 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20112080 con 0x7fdd201031e0 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd25805640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20112380 con 0x7fdd20103b00 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 0x7fdd201018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 0x7fdd201018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58516/0 (socket says 172.21.15.37:58516) 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd1f7fe640 1 -- 172.21.15.37:0/419092288 learned_addr learned my addr 172.21.15.37:0/419092288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd20103b00 0x7fdd20101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.295+0000 7fdd1effd640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd201031e0 0x7fdd20100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd1f7fe640 1 -- 172.21.15.37:0/419092288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd20103b00 msgr2=0x7fdd20101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd1f7fe640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd20103b00 0x7fdd20101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd1f7fe640 1 -- 172.21.15.37:0/419092288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd201031e0 msgr2=0x7fdd20100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd1f7fe640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd201031e0 0x7fdd20100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd1f7fe640 1 -- 172.21.15.37:0/419092288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd201a6c20 con 0x7fdd2010c5b0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd1f7fe640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 0x7fdd201018b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800d720 tx=0x7fdd1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd24803640 1 -- 172.21.15.37:0/419092288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18017070 con 0x7fdd2010c5b0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd24803640 1 -- 172.21.15.37:0/419092288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1800de30 con 0x7fdd2010c5b0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd201a6f10 con 0x7fdd2010c5b0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd24803640 1 -- 172.21.15.37:0/419092288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18004d00 con 0x7fdd2010c5b0 2024-04-23T18:32:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.296+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201a7430 con 0x7fdd2010c5b0 2024-04-23T18:32:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.298+0000 7fdd24803640 1 -- 172.21.15.37:0/419092288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd18004ea0 con 0x7fdd2010c5b0 2024-04-23T18:32:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 Processor -- start 2024-04-23T18:32:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 -- start start 2024-04-23T18:32:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4620 0x555a04ff4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4f40 0x555a04fff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 0x555a05001bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555a04f59970 con 0x555a04fff800 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555a04ef64a0 con 0x555a04ff4620 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd285fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555a04f55930 con 0x555a04ff4f40 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd24803640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fdd080a3e40 0x7fdd080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 0x555a05001bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 0x555a05001bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58528/0 (socket says 172.21.15.37:58528) 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/2975649387 learned_addr learned my addr 172.21.15.37:0/2975649387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1f7fe640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fdd080a3e40 0x7fdd080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1effd640 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4f40 0x555a04fff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4620 0x555a04ff4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd24803640 1 -- 172.21.15.37:0/419092288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7fdd180d0120 con 0x7fdd2010c5b0 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/2975649387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4f40 msgr2=0x555a04fff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4f40 0x555a04fff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/2975649387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4620 msgr2=0x555a04ff4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.299+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4620 0x555a04ff4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/2975649387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a04c6c210 con 0x555a04fff800 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 0x555a05001bf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdd140149c0 tx=0x7fdd1400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c628e7a4b65f7ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd1f7fe640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fdd080a3e40 0x7fdd080a6300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cd8e0 tx=0x7fdd18091510 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd05ffb640 1 -- 172.21.15.37:0/2975649387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd140142d0 con 0x555a04fff800 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd05ffb640 1 -- 172.21.15.37:0/2975649387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd140054d0 con 0x555a04fff800 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd05ffb640 1 -- 172.21.15.37:0/2975649387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400f760 con 0x555a04fff800 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.300+0000 7fdd285fc740 1 -- 172.21.15.37:0/2975649387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 msgr2=0x555a05001bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 0x555a05001bf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdd140149c0 tx=0x7fdd1400c4d0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 -- 172.21.15.37:0/2975649387 shutdown_connections 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04fff800 0x555a05001bf0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4f40 0x555a04fff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 --2- 172.21.15.37:0/2975649387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4620 0x555a04ff4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 -- 172.21.15.37:0/2975649387 >> 172.21.15.37:0/2975649387 conn(0x555a04ff0b30 msgr2=0x555a04ff0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 -- 172.21.15.37:0/2975649387 shutdown_connections 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 -- 172.21.15.37:0/2975649387 wait complete. 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 -- start start 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.301+0000 7fdd285fc740 1 Processor -- start 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 0x555a04fff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4f40 0x555a04ffc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04fff800 0x555a04ffc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555a05002430 con 0x555a04ff4620 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555a04ef40b0 con 0x555a04ff4f40 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555a050025b0 con 0x555a04fff800 2024-04-23T18:32:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 10 client.0 _ll_register_callbacks cb 0x555a04ee1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 0x555a04fff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 0x555a04fff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58538/0 (socket says 172.21.15.37:58538) 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/3300193070 learned_addr learned my addr 172.21.15.37:0/3300193070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1effd640 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4f40 0x555a04ffc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/3300193070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04fff800 msgr2=0x555a04ffc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04fff800 0x555a04ffc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4f40 msgr2=0x555a04ffc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4f40 0x555a04ffc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 -- 172.21.15.37:0/3300193070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a04d38620 con 0x555a04ff4620 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 0x555a04fff090 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdd140058c0 tx=0x7fdd1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8897e2b334eb7865 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdcfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3300193070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14027020 con 0x555a04ff4620 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3300193070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7fdd14004e60 con 0x555a04ff4620 2024-04-23T18:32:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.302+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd140054d0 con 0x555a04ff4620 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3300193070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdd14032050 con 0x555a04ff4620 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 10 client.16839 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555a0500e310 con 0x555a04ff4620 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3300193070 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd140054d0 con 0x555a04ff4620 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3300193070 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7fdd14012450 con 0x555a04ff4620 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 10 client.16839 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 20 client.16839 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a0500f770 0x555a05011b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 4 client.16839 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a04f67390 con 0x555a0500f770 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.303+0000 7fdd1effd640 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a0500f770 0x555a05011b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.304+0000 7fdcfeffd640 10 client.16839 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.304+0000 7fdd1effd640 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a0500f770 0x555a05011b60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207febcadaa5bb9 server_cookie=8013fdfbc0c16c1e in_seq=0 out_seq=0 2024-04-23T18:32:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.304+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3300193070 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a04f67390 con 0x555a0500f770 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.304+0000 7fdcfeffd640 10 client.16839 handle_command_reply: tid=0 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.304+0000 7fdd285fc740 1 client.16839 shutdown 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 msgr2=0x555a04fff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 0x555a04fff090 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdd140058c0 tx=0x7fdd1400ce30 comp rx=0 tx=0).stop 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 shutdown_connections 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a0500f770 0x555a05011b60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04fff800 0x555a04ffc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04ff4f40 0x555a04ffc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3300193070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4620 0x555a04fff090 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdcfeffd640 0 client.16839 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 >> 172.21.15.37:0/3300193070 conn(0x555a04ff0b30 msgr2=0x555a04ff6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 shutdown_connections 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 1 -- 172.21.15.37:0/3300193070 wait complete. 2024-04-23T18:32:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.305+0000 7fdd285fc740 20 client.16839 trim_cache size 0 max 16384 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 Processor -- start 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 -- start start 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a0500b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a04fefef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05017160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201b2710 con 0x555a04ff4f40 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201affb0 con 0x555a04fff800 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd285fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201ade20 con 0x555a04ff4620 2024-04-23T18:32:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd1effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a04fefef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd1effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a04fefef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58554/0 (socket says 172.21.15.37:58554) 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.320+0000 7fdd1effd640 1 -- 172.21.15.37:0/1972084205 learned_addr learned my addr 172.21.15.37:0/1972084205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05017160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1effd640 1 -- 172.21.15.37:0/1972084205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 msgr2=0x555a0500b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1effd640 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a0500b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1effd640 1 -- 172.21.15.37:0/1972084205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 msgr2=0x555a05017160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1effd640 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05017160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1effd640 1 -- 172.21.15.37:0/1972084205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a04ffcec0 con 0x555a04ff4f40 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05017160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd1effd640 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a04fefef0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd10009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a55661feff39a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd057fa640 1 -- 172.21.15.37:0/1972084205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10034040 con 0x555a04ff4f40 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd057fa640 1 -- 172.21.15.37:0/1972084205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1004c070 con 0x555a04ff4f40 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.321+0000 7fdd057fa640 1 -- 172.21.15.37:0/1972084205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1002fd10 con 0x555a04ff4f40 2024-04-23T18:32:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 -- 172.21.15.37:0/1972084205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 msgr2=0x555a04fefef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a04fefef0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd10009680 comp rx=0 tx=0).stop 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 -- 172.21.15.37:0/1972084205 shutdown_connections 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05017160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a04fefef0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 --2- 172.21.15.37:0/1972084205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a0500b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 -- 172.21.15.37:0/1972084205 >> 172.21.15.37:0/1972084205 conn(0x555a04ff0b30 msgr2=0x555a04ffc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 -- 172.21.15.37:0/1972084205 shutdown_connections 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 -- 172.21.15.37:0/1972084205 wait complete. 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 -- start start 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.322+0000 7fdd285fc740 1 Processor -- start 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a05007c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a05004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05005510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201ae470 con 0x555a04ff4f40 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201af8e0 con 0x555a04fff800 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201aeb40 con 0x555a04ff4620 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 10 client.0 _ll_register_callbacks cb 0x7fdd140148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a05004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a05004fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58558/0 (socket says 172.21.15.37:58558) 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 -- 172.21.15.37:0/3395448726 learned_addr learned my addr 172.21.15.37:0/3395448726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05005510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a05007c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 -- 172.21.15.37:0/3395448726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 msgr2=0x555a05007c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a05007c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 -- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 msgr2=0x555a05005510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05005510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 -- 172.21.15.37:0/3395448726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a05005d10 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd1effd640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a05004fd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006010 tx=0x7fdd1002f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5eb268b2af2362 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdcfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3395448726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10034070 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3395448726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7fdd1002fc90 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.323+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd1004c070 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3395448726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdd1002fe90 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 10 client.16845 fetch_fsmap learned FSMap version 43 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555a05075990 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3395448726 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1004c070 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3395448726 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8350+0+0 (secure 0 0 0) 0x7fdd10030b30 con 0x555a04ff4f40 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 10 client.16845 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 20 client.16845 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a050760a0 0x555a05078490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 4 client.16845 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a04f61200 con 0x555a050760a0 2024-04-23T18:32:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.324+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a050760a0 0x555a05078490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.325+0000 7fdd1e7fc640 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a050760a0 0x555a05078490 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e5d4908438d137 server_cookie=d1a969294bcc0a0c in_seq=0 out_seq=0 2024-04-23T18:32:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.325+0000 7fdcfeffd640 10 client.16845 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:46.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:46 smithi172 ceph-mon[32456]: pgmap v385: 97 pgs: 97 active+clean; 447 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:32:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:46 smithi173 ceph-mon[32906]: pgmap v385: 97 pgs: 97 active+clean; 447 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:32:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.367+0000 7fdcfeffd640 1 -- 172.21.15.37:0/3395448726 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a04f61200 con 0x555a050760a0 2024-04-23T18:32:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.367+0000 7fdcfeffd640 10 client.16845 handle_command_reply: tid=0 2024-04-23T18:32:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 client.16845 shutdown 2024-04-23T18:32:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 msgr2=0x555a05004fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a05004fd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006010 tx=0x7fdd1002f870 comp rx=0 tx=0).stop 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 shutdown_connections 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x555a050760a0 0x555a05078490 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555a04fff800 0x555a05005510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a04ff4f40 0x555a05004fd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdcfeffd640 0 client.16845 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 --2- 172.21.15.37:0/3395448726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555a04ff4620 0x555a05007c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 >> 172.21.15.37:0/3395448726 conn(0x555a04ff0b30 msgr2=0x555a0500a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 shutdown_connections 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 1 -- 172.21.15.37:0/3395448726 wait complete. 2024-04-23T18:32:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd285fc740 20 client.16845 trim_cache size 0 max 16384 2024-04-23T18:32:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:32:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:32:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:32:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.368+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fdd080a3e40 msgr2=0x7fdd080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fdd080a3e40 0x7fdd080a6300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cd8e0 tx=0x7fdd18091510 comp rx=0 tx=0).stop 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 msgr2=0x7fdd201018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 0x7fdd201018b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800d720 tx=0x7fdd1800dbe0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 shutdown_connections 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fdd080a3e40 0x7fdd080a6300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd2010c5b0 0x7fdd201018b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd20103b00 0x7fdd20101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 --2- 172.21.15.37:0/419092288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd201031e0 0x7fdd20100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 >> 172.21.15.37:0/419092288 conn(0x7fdd20077fd0 msgr2=0x7fdd200fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 shutdown_connections 2024-04-23T18:32:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:46.369+0000 7fdd25805640 1 -- 172.21.15.37:0/419092288 wait complete. 2024-04-23T18:32:46.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:46 smithi037 ceph-mon[29404]: pgmap v385: 97 pgs: 97 active+clean; 447 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:32:46.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 599710 3.69 MB/sec execute 531 sec latency 1840.234 ms 2024-04-23T18:32:47.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 601129 3.69 MB/sec execute 532 sec latency 13.750 ms 2024-04-23T18:32:48.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:48 smithi172 ceph-mon[32456]: pgmap v386: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-04-23T18:32:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:48 smithi173 ceph-mon[32906]: pgmap v386: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-04-23T18:32:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:48 smithi037 ceph-mon[29404]: pgmap v386: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-04-23T18:32:48.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 602705 3.70 MB/sec execute 533 sec latency 213.065 ms 2024-04-23T18:32:49.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:49 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:32:49.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:49 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:49.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:49 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:49 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:49 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.337 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:49 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:49 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:32:49.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:49 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:49.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:49 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:49 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:49 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:49 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:49 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:32:49.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:49 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:49.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:49 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:49 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:49 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:49 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:49.854 INFO:tasks.workunit.client.0.smithi037.stdout: 1 604369 3.71 MB/sec execute 534 sec latency 11.478 ms 2024-04-23T18:32:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:50 smithi172 ceph-mon[32456]: pgmap v387: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-23T18:32:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:50 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:50 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:50 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:50 smithi173 ceph-mon[32906]: pgmap v387: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-23T18:32:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:50 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:50 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:50 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:50 smithi037 ceph-mon[29404]: pgmap v387: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-23T18:32:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:50 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:50 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:50 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:32:50.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 605495 3.72 MB/sec execute 535 sec latency 324.273 ms 2024-04-23T18:32:51.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 605495 3.71 MB/sec execute 536 sec latency 1324.374 ms 2024-04-23T18:32:52.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:52 smithi172 ceph-mon[32456]: pgmap v388: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-23T18:32:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:52 smithi173 ceph-mon[32906]: pgmap v388: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-23T18:32:52.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:52 smithi037 ceph-mon[29404]: pgmap v388: 97 pgs: 97 active+clean; 453 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-04-23T18:32:52.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 606222 3.71 MB/sec execute 537 sec latency 2056.053 ms 2024-04-23T18:32:53.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 607898 3.72 MB/sec execute 538 sec latency 21.727 ms 2024-04-23T18:32:54.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:54 smithi172 ceph-mon[32456]: pgmap v389: 97 pgs: 97 active+clean; 461 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-04-23T18:32:54.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:54 smithi173 ceph-mon[32906]: pgmap v389: 97 pgs: 97 active+clean; 461 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-04-23T18:32:54.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:54 smithi037 ceph-mon[29404]: pgmap v389: 97 pgs: 97 active+clean; 461 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-04-23T18:32:54.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 609036 3.73 MB/sec execute 539 sec latency 315.310 ms 2024-04-23T18:32:55.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 609036 3.72 MB/sec execute 540 sec latency 1315.408 ms 2024-04-23T18:32:56.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:56 smithi172 ceph-mon[32456]: pgmap v390: 97 pgs: 97 active+clean; 461 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-04-23T18:32:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:56 smithi173 ceph-mon[32906]: pgmap v390: 97 pgs: 97 active+clean; 461 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-04-23T18:32:56.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:56 smithi037 ceph-mon[29404]: pgmap v390: 97 pgs: 97 active+clean; 461 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-04-23T18:32:56.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 609036 3.71 MB/sec execute 541 sec latency 2315.495 ms 2024-04-23T18:32:57.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 609036 3.70 MB/sec execute 542 sec latency 3315.574 ms 2024-04-23T18:32:58.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:32:58 smithi172 ceph-mon[32456]: pgmap v391: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-04-23T18:32:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:32:58 smithi173 ceph-mon[32906]: pgmap v391: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-04-23T18:32:58.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:32:58 smithi037 ceph-mon[29404]: pgmap v391: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-04-23T18:32:58.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 609812 3.70 MB/sec execute 543 sec latency 4019.930 ms 2024-04-23T18:32:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 -- 172.21.15.37:0/3051778456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d0103b00 msgr2=0x7f61d010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/3051778456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d0103b00 0x7f61d010bfe0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009900 tx=0x7f61b802a510 comp rx=0 tx=0).stop 2024-04-23T18:32:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 -- 172.21.15.37:0/3051778456 shutdown_connections 2024-04-23T18:32:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/3051778456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d010c520 0x7f61d010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/3051778456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d0103b00 0x7f61d010bfe0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/3051778456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d01031e0 0x7f61d01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 -- 172.21.15.37:0/3051778456 >> 172.21.15.37:0/3051778456 conn(0x7f61d0077fd0 msgr2=0x7f61d00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.462+0000 7f61d4b9b640 1 -- 172.21.15.37:0/3051778456 shutdown_connections 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.463+0000 7f61d4b9b640 1 -- 172.21.15.37:0/3051778456 wait complete. 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.463+0000 7f61d4b9b640 1 Processor -- start 2024-04-23T18:32:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.463+0000 7f61d4b9b640 1 -- start start 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.463+0000 7f61d4b9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 0x7f61d0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.463+0000 7f61d4b9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d0103b00 0x7f61d0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61d4b9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d010c520 0x7f61d019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61d4b9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01121b0 con 0x7f61d01031e0 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61d4b9b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0112030 con 0x7f61d010c520 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61d4b9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0112330 con 0x7f61d0103b00 2024-04-23T18:32:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 0x7f61d0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 0x7f61d0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53350/0 (socket says 172.21.15.37:53350) 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 -- 172.21.15.37:0/385819490 learned_addr learned my addr 172.21.15.37:0/385819490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ced9d640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d010c520 0x7f61d019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61c7fff640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d0103b00 0x7f61d0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 -- 172.21.15.37:0/385819490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d0103b00 msgr2=0x7f61d0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d0103b00 0x7f61d0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 -- 172.21.15.37:0/385819490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d010c520 msgr2=0x7f61d019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d010c520 0x7f61d019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 -- 172.21.15.37:0/385819490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d019d6b0 con 0x7f61d01031e0 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61ce59c640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 0x7f61d0198710 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9f0 tx=0x7f61c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61c77fe640 1 -- 172.21.15.37:0/385819490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c000cd80 con 0x7f61d01031e0 2024-04-23T18:32:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d019d9a0 con 0x7f61d01031e0 2024-04-23T18:32:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61c77fe640 1 -- 172.21.15.37:0/385819490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c00108e0 con 0x7f61d01031e0 2024-04-23T18:32:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d01a5230 con 0x7f61d01031e0 2024-04-23T18:32:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.464+0000 7f61c77fe640 1 -- 172.21.15.37:0/385819490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0018920 con 0x7f61d01031e0 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.467+0000 7f61c77fe640 1 -- 172.21.15.37:0/385819490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f61c0010430 con 0x7f61d01031e0 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.467+0000 7f61d7991740 1 Processor -- start 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.467+0000 7f61d7991740 1 -- start start 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61d7991740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18620 0x55d1e6d18a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61d7991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61c77fe640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f61bc0a3dc0 0x7f61bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61d7991740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d23800 0x55d1e6d25bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61d7991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d1e6c7d970 con 0x55d1e6d18f40 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61d7991740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1e6c1a4a0 con 0x55d1e6d23800 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61d7991740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d1e6c79930 con 0x55d1e6d18620 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61c7fff640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f61bc0a3dc0 0x7f61bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53360/0 (socket says 172.21.15.37:53360) 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 -- 172.21.15.37:0/3484290238 learned_addr learned my addr 172.21.15.37:0/3484290238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ced9d640 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18620 0x55d1e6d18a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61c77fe640 1 -- 172.21.15.37:0/385819490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f61c00d0b20 con 0x7f61d01031e0 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ced9d640 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d23800 0x55d1e6d25bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 -- 172.21.15.37:0/3484290238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18620 msgr2=0x55d1e6d18a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18620 0x55d1e6d18a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 -- 172.21.15.37:0/3484290238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d23800 msgr2=0x55d1e6d25bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d23800 0x55d1e6d25bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61ce59c640 1 -- 172.21.15.37:0/3484290238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e6990210 con 0x55d1e6d18f40 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.468+0000 7f61c7fff640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f61bc0a3dc0 0x7f61bc0a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61b8031440 tx=0x7f61b8005b50 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61ce59c640 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d232c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9c0 tx=0x7f61c006e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5173f6d862d77e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61c4ff9640 1 -- 172.21.15.37:0/3484290238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0086070 con 0x55d1e6d18f40 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61c4ff9640 1 -- 172.21.15.37:0/3484290238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c00bd0d0 con 0x55d1e6d18f40 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61c4ff9640 1 -- 172.21.15.37:0/3484290238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00ce4b0 con 0x55d1e6d18f40 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 -- 172.21.15.37:0/3484290238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 msgr2=0x55d1e6d232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d232c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9c0 tx=0x7f61c006e630 comp rx=0 tx=0).stop 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 -- 172.21.15.37:0/3484290238 shutdown_connections 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d23800 0x55d1e6d25bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d232c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 --2- 172.21.15.37:0/3484290238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18620 0x55d1e6d18a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.469+0000 7f61d7991740 1 -- 172.21.15.37:0/3484290238 >> 172.21.15.37:0/3484290238 conn(0x55d1e6d14b30 msgr2=0x55d1e6d14f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 -- 172.21.15.37:0/3484290238 shutdown_connections 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 -- 172.21.15.37:0/3484290238 wait complete. 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 -- start start 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 Processor -- start 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d23800 0x55d1e6d20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d1e6d26430 con 0x55d1e6d18f40 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1e6c180b0 con 0x55d1e6d18620 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d1e6d265b0 con 0x55d1e6d23800 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61d7991740 10 client.0 _ll_register_callbacks cb 0x55d1e6c05d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d204c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53370/0 (socket says 172.21.15.37:53370) 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61c7fff640 1 -- 172.21.15.37:0/3517971299 learned_addr learned my addr 172.21.15.37:0/3517971299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.470+0000 7f61ced9d640 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d23800 0x55d1e6d20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61ce59c640 1 -- 172.21.15.37:0/3517971299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d23800 msgr2=0x55d1e6d20ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61ce59c640 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d23800 0x55d1e6d20ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61ce59c640 1 -- 172.21.15.37:0/3517971299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 msgr2=0x55d1e6d204c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61ce59c640 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d204c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61ce59c640 1 -- 172.21.15.37:0/3517971299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e6a5c620 con 0x55d1e6d18620 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61ce59c640 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d231d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9c0 tx=0x7f61c0009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147cc4f8c4dcb1a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61b5ffb640 1 -- 172.21.15.37:0/3517971299 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0086070 con 0x55d1e6d18620 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c00bd0d0 con 0x55d1e6d18620 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61b5ffb640 1 -- 172.21.15.37:0/3517971299 <== mon.2 v2:172.21.15.172:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f61c0087300 con 0x55d1e6d18620 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61b5ffb640 1 -- 172.21.15.37:0/3517971299 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f61c00a0e30 con 0x55d1e6d18620 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61d7991740 10 client.26240 fetch_fsmap learned FSMap version 44 2024-04-23T18:32:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d1e6d323c0 con 0x55d1e6d18620 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.471+0000 7f61b5ffb640 1 -- 172.21.15.37:0/3517971299 <== mon.2 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c00bd0d0 con 0x55d1e6d18620 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61b5ffb640 1 -- 172.21.15.37:0/3517971299 <== mon.2 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f61c00b9070 con 0x55d1e6d18620 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61d7991740 10 client.26240 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61d7991740 10 client.26240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61d7991740 20 client.26240 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61d7991740 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d33710 0x55d1e6d35b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61d7991740 4 client.26240 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e6c8b390 con 0x55d1e6d33710 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61c7fff640 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d33710 0x55d1e6d35b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61b5ffb640 10 client.26240 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.472+0000 7f61c7fff640 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d33710 0x55d1e6d35b00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d346ed44ddee4e27 server_cookie=c9eed09b93cb3f12 in_seq=0 out_seq=0 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61b5ffb640 1 -- 172.21.15.37:0/3517971299 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1e6c8b390 con 0x55d1e6d33710 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61b5ffb640 10 client.26240 handle_command_reply: tid=0 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 client.26240 shutdown 2024-04-23T18:32:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 msgr2=0x55d1e6d231d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d231d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9c0 tx=0x7f61c0009700 comp rx=0 tx=0).stop 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 shutdown_connections 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d33710 0x55d1e6d35b00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d23800 0x55d1e6d20ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d18f40 0x55d1e6d204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 --2- 172.21.15.37:0/3517971299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d231d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61b5ffb640 0 client.26240 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 >> 172.21.15.37:0/3517971299 conn(0x55d1e6d14b30 msgr2=0x55d1e6d1a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 shutdown_connections 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 1 -- 172.21.15.37:0/3517971299 wait complete. 2024-04-23T18:32:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.473+0000 7f61d7991740 20 client.26240 trim_cache size 0 max 16384 2024-04-23T18:32:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 Processor -- start 2024-04-23T18:32:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 -- start start 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d2f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d3b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01ae2f0 con 0x55d1e6d23800 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01abb90 con 0x55d1e6d18620 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.489+0000 7f61d7991740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01a9a00 con 0x55d1e6d18f40 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61ced9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d3b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61ced9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d3b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53386/0 (socket says 172.21.15.37:53386) 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61ced9d640 1 -- 172.21.15.37:0/1689018896 learned_addr learned my addr 172.21.15.37:0/1689018896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61ce59c640 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d2f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 -- 172.21.15.37:0/1689018896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 msgr2=0x55d1e6d2f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d2f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 -- 172.21.15.37:0/1689018896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 msgr2=0x55d1e6d3b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d3b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 -- 172.21.15.37:0/1689018896 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e6d25e20 con 0x55d1e6d18f40 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61ced9d640 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d3b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61c7fff640 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d3b100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61b80057c0 tx=0x7f61b8040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b793a29d9ce9b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61b7fff640 1 -- 172.21.15.37:0/1689018896 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8035040 con 0x55d1e6d18f40 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61b7fff640 1 -- 172.21.15.37:0/1689018896 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b8005040 con 0x55d1e6d18f40 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.490+0000 7f61b7fff640 1 -- 172.21.15.37:0/1689018896 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8050360 con 0x55d1e6d18f40 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 -- 172.21.15.37:0/1689018896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 msgr2=0x55d1e6d3b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d3b100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61b80057c0 tx=0x7f61b8040000 comp rx=0 tx=0).stop 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 -- 172.21.15.37:0/1689018896 shutdown_connections 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d3b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d3b100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 --2- 172.21.15.37:0/1689018896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d2f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 -- 172.21.15.37:0/1689018896 >> 172.21.15.37:0/1689018896 conn(0x55d1e6d14b30 msgr2=0x55d1e6d13cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 -- 172.21.15.37:0/1689018896 shutdown_connections 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.491+0000 7f61d7991740 1 -- 172.21.15.37:0/1689018896 wait complete. 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 -- start start 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 Processor -- start 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d29290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01aa050 con 0x55d1e6d23800 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01ab4c0 con 0x55d1e6d18620 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61d01aa720 con 0x55d1e6d18f40 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61d7991740 10 client.0 _ll_register_callbacks cb 0x7f61c009c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d29d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53394/0 (socket says 172.21.15.37:53394) 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 -- 172.21.15.37:0/177619299 learned_addr learned my addr 172.21.15.37:0/177619299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:32:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d29290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61c7fff640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 -- 172.21.15.37:0/177619299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 msgr2=0x55d1e6d297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 -- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 msgr2=0x55d1e6d29290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d29290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ced9d640 1 -- 172.21.15.37:0/177619299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e6d989d0 con 0x55d1e6d23800 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61c7fff640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.492+0000 7f61ce59c640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61ced9d640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d29d10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f61c8011970 tx=0x7f61c8011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb22385f3ac04950 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61b5ffb640 1 -- 172.21.15.37:0/177619299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8023030 con 0x55d1e6d23800 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61b5ffb640 1 -- 172.21.15.37:0/177619299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f61c8010c00 con 0x55d1e6d23800 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b8005040 con 0x55d1e6d23800 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61b5ffb640 1 -- 172.21.15.37:0/177619299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f61c8010e40 con 0x55d1e6d23800 2024-04-23T18:32:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 10 client.16863 fetch_fsmap learned FSMap version 44 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d1e6d99ce0 con 0x55d1e6d23800 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61b5ffb640 1 -- 172.21.15.37:0/177619299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b8005040 con 0x55d1e6d23800 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61b5ffb640 1 -- 172.21.15.37:0/177619299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f61c801f470 con 0x55d1e6d23800 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 10 client.16863 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 20 client.16863 populate_metadata read hostname 'smithi037' 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d9a530 0x55d1e6d9c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 4 client.16863 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.493+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1e6c85200 con 0x55d1e6d9a530 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.494+0000 7f61ce59c640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d9a530 0x55d1e6d9c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.494+0000 7f61ce59c640 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d9a530 0x55d1e6d9c950 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c909c91c1ec5728 server_cookie=7eb67ac0019185b8 in_seq=0 out_seq=0 2024-04-23T18:32:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.494+0000 7f61b5ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61b5ffb640 1 -- 172.21.15.37:0/177619299 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1e6c85200 con 0x55d1e6d9a530 2024-04-23T18:32:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61b5ffb640 10 client.16863 handle_command_reply: tid=0 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 client.16863 shutdown 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 msgr2=0x55d1e6d29d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d29d10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f61c8011970 tx=0x7f61c8011d70 comp rx=0 tx=0).stop 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 shutdown_connections 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55d1e6d9a530 0x55d1e6d9c950 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1e6d23800 0x55d1e6d29d10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1e6d18f40 0x55d1e6d297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61b5ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 --2- 172.21.15.37:0/177619299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1e6d18620 0x55d1e6d29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 >> 172.21.15.37:0/177619299 conn(0x55d1e6d14b30 msgr2=0x55d1e6d2e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 shutdown_connections 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 1 -- 172.21.15.37:0/177619299 wait complete. 2024-04-23T18:32:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.531+0000 7f61d7991740 20 client.16863 trim_cache size 0 max 16384 2024-04-23T18:32:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:32:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:32:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:32:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f61bc0a3dc0 msgr2=0x7f61bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f61bc0a3dc0 0x7f61bc0a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61b8031440 tx=0x7f61b8005b50 comp rx=0 tx=0).stop 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 msgr2=0x7f61d0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 0x7f61d0198710 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f61c000e9f0 tx=0x7f61c000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 shutdown_connections 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f61bc0a3dc0 0x7f61bc0a6280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d010c520 0x7f61d019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d0103b00 0x7f61d0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 --2- 172.21.15.37:0/385819490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61d01031e0 0x7f61d0198710 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 >> 172.21.15.37:0/385819490 conn(0x7f61d0077fd0 msgr2=0x7f61d00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 shutdown_connections 2024-04-23T18:32:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:32:59.532+0000 7f61d4b9b640 1 -- 172.21.15.37:0/385819490 wait complete. 2024-04-23T18:32:59.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 611562 3.72 MB/sec execute 544 sec latency 17.986 ms 2024-04-23T18:33:00.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:00 smithi172 ceph-mon[32456]: pgmap v392: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-04-23T18:33:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:00 smithi173 ceph-mon[32906]: pgmap v392: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-04-23T18:33:00.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:00 smithi037 ceph-mon[29404]: pgmap v392: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-04-23T18:33:00.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 612586 3.72 MB/sec execute 545 sec latency 398.611 ms 2024-04-23T18:33:01.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 612586 3.71 MB/sec execute 546 sec latency 1398.644 ms 2024-04-23T18:33:02.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:02 smithi172 ceph-mon[32456]: pgmap v393: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-04-23T18:33:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:02 smithi173 ceph-mon[32906]: pgmap v393: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-04-23T18:33:02.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:02 smithi037 ceph-mon[29404]: pgmap v393: 97 pgs: 97 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-04-23T18:33:02.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 613166 3.71 MB/sec execute 547 sec latency 2127.511 ms 2024-04-23T18:33:03.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 614722 3.72 MB/sec execute 548 sec latency 22.659 ms 2024-04-23T18:33:04.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:04 smithi173 ceph-mon[32906]: pgmap v394: 97 pgs: 97 active+clean; 470 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-23T18:33:04.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:04 smithi037 ceph-mon[29404]: pgmap v394: 97 pgs: 97 active+clean; 470 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-23T18:33:04.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:04 smithi172 ceph-mon[32456]: pgmap v394: 97 pgs: 97 active+clean; 470 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2024-04-23T18:33:04.855 INFO:tasks.workunit.client.0.smithi037.stdout: 1 615956 3.73 MB/sec execute 549 sec latency 27.134 ms 2024-04-23T18:33:05.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:05 smithi173 ceph-mon[32906]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:33:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:05 smithi037 ceph-mon[29404]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:33:05.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:05 smithi172 ceph-mon[32456]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:33:05.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 616129 3.72 MB/sec execute 550 sec latency 903.958 ms 2024-04-23T18:33:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:06 smithi173 ceph-mon[32906]: pgmap v396: 97 pgs: 97 active+clean; 470 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-04-23T18:33:06.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:06 smithi037 ceph-mon[29404]: pgmap v396: 97 pgs: 97 active+clean; 470 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-04-23T18:33:06.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:06 smithi172 ceph-mon[32456]: pgmap v396: 97 pgs: 97 active+clean; 470 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-04-23T18:33:06.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 616129 3.71 MB/sec execute 551 sec latency 1904.062 ms 2024-04-23T18:33:07.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 616129 3.71 MB/sec execute 552 sec latency 2904.100 ms 2024-04-23T18:33:08.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:08 smithi037 ceph-mon[29404]: pgmap v397: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:08.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:08 smithi172 ceph-mon[32456]: pgmap v397: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:08.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:08 smithi173 ceph-mon[32906]: pgmap v397: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:08.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 616905 3.70 MB/sec execute 553 sec latency 3606.562 ms 2024-04-23T18:33:09.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:09 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:33:09.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:09 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:33:09.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:09 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:33:09.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 618660 3.72 MB/sec execute 554 sec latency 20.888 ms 2024-04-23T18:33:10.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:10 smithi037 ceph-mon[29404]: pgmap v398: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:10.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:10 smithi172 ceph-mon[32456]: pgmap v398: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:10.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:10 smithi173 ceph-mon[32906]: pgmap v398: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:10.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 619675 3.72 MB/sec execute 555 sec latency 461.294 ms 2024-04-23T18:33:11.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 619675 3.71 MB/sec execute 556 sec latency 1461.382 ms 2024-04-23T18:33:12.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:12 smithi172 ceph-mon[32456]: pgmap v399: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:12.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:12 smithi172 ceph-mon[32456]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:33:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:12 smithi173 ceph-mon[32906]: pgmap v399: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:12.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:12 smithi173 ceph-mon[32906]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:33:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:12 smithi037 ceph-mon[29404]: pgmap v399: 97 pgs: 97 active+clean; 474 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-04-23T18:33:12.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:12 smithi037 ceph-mon[29404]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:33:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 -- 172.21.15.37:0/2735054172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c106170 msgr2=0x7fc08c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2735054172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c106170 0x7fc08c10dd40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc084009ca0 tx=0x7fc08402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 -- 172.21.15.37:0/2735054172 shutdown_connections 2024-04-23T18:33:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2735054172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c106170 0x7fc08c10dd40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2735054172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c101aa0 0x7fc08c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2735054172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc08c1010f0 0x7fc08c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 -- 172.21.15.37:0/2735054172 >> 172.21.15.37:0/2735054172 conn(0x7fc08c077fd0 msgr2=0x7fc08c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 -- 172.21.15.37:0/2735054172 shutdown_connections 2024-04-23T18:33:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.639+0000 7fc0910cf640 1 -- 172.21.15.37:0/2735054172 wait complete. 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 Processor -- start 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 -- start start 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 0x7fc08c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 0x7fc08c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc08c19cf20 0x7fc08c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1121f0 con 0x7fc08c1010f0 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c112070 con 0x7fc08c19cf20 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc0910cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c112370 con 0x7fc08c106170 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc08ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 0x7fc08c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc08259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 0x7fc08c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc08259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 0x7fc08c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.37:56306/0 (socket says 172.21.15.37:56306) 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc08259c640 1 -- 172.21.15.37:0/2885614983 learned_addr learned my addr 172.21.15.37:0/2885614983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc08ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 0x7fc08c1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43878/0 (socket says 172.21.15.37:43878) 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.640+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc08c19cf20 0x7fc08c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08259c640 1 -- 172.21.15.37:0/2885614983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc08c19cf20 msgr2=0x7fc08c19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08259c640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc08c19cf20 0x7fc08c19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08259c640 1 -- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 msgr2=0x7fc08c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08259c640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 0x7fc08c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08259c640 1 -- 172.21.15.37:0/2885614983 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c19da10 con 0x7fc08c106170 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 0x7fc08c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc08259c640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 0x7fc08c198bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc07400e9f0 tx=0x7fc07400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc088d99640 1 -- 172.21.15.37:0/2885614983 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07400ccc0 con 0x7fc08c106170 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc088d99640 1 -- 172.21.15.37:0/2885614983 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0740054d0 con 0x7fc08c106170 2024-04-23T18:33:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c19dd00 con 0x7fc08c106170 2024-04-23T18:33:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc088d99640 1 -- 172.21.15.37:0/2885614983 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074010430 con 0x7fc08c106170 2024-04-23T18:33:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.641+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a54a0 con 0x7fc08c106170 2024-04-23T18:33:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.644+0000 7fc088d99640 1 -- 172.21.15.37:0/2885614983 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc0740046c0 con 0x7fc08c106170 2024-04-23T18:33:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 Processor -- start 2024-04-23T18:33:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 -- start start 2024-04-23T18:33:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 0x55cf452e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf452f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf4524b970 con 0x55cf452e6620 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf451e84a0 con 0x55cf452f1800 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc093de7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cf45247930 con 0x55cf452e6f40 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 0x55cf452e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 0x55cf452e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43888/0 (socket says 172.21.15.37:43888) 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc08b59e640 1 -- 172.21.15.37:0/2842982032 learned_addr learned my addr 172.21.15.37:0/2842982032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc08259c640 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf452f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.645+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc088d99640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0780a3dc0 0x7fc0780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 -- 172.21.15.37:0/2842982032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 msgr2=0x55cf452f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 -- 172.21.15.37:0/2842982032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 msgr2=0x55cf452f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf452f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 -- 172.21.15.37:0/2842982032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf44f5e210 con 0x55cf452e6620 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08259c640 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf452f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc088d99640 1 -- 172.21.15.37:0/2885614983 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fc0740d0b70 con 0x7fc08c106170 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0780a3dc0 0x7fc0780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 0x55cf452e6a00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc084005fa0 tx=0x7fc084009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de59b9a7aff41721 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08159a640 1 -- 172.21.15.37:0/2842982032 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084042040 con 0x55cf452e6620 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08159a640 1 -- 172.21.15.37:0/2842982032 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084036070 con 0x55cf452e6620 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08159a640 1 -- 172.21.15.37:0/2842982032 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084004700 con 0x55cf452e6620 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.646+0000 7fc08b59e640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0780a3dc0 0x7fc0780a6280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc08404d000 tx=0x7fc08404e000 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 -- 172.21.15.37:0/2842982032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 msgr2=0x55cf452e6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 0x55cf452e6a00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc084005fa0 tx=0x7fc084009800 comp rx=0 tx=0).stop 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 -- 172.21.15.37:0/2842982032 shutdown_connections 2024-04-23T18:33:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf452f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 --2- 172.21.15.37:0/2842982032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6620 0x55cf452e6a00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 -- 172.21.15.37:0/2842982032 >> 172.21.15.37:0/2842982032 conn(0x55cf452e2b30 msgr2=0x55cf452e2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 -- 172.21.15.37:0/2842982032 shutdown_connections 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.647+0000 7fc093de7740 1 -- 172.21.15.37:0/2842982032 wait complete. 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 -- start start 2024-04-23T18:33:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 Processor -- start 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf452ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf452f4430 con 0x55cf452f1800 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf451e60b0 con 0x55cf452e6620 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cf452f45b0 con 0x55cf452e6f40 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc093de7740 10 client.0 _ll_register_callbacks cb 0x55cf451d3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf452ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf452ee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43898/0 (socket says 172.21.15.37:43898) 2024-04-23T18:33:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 -- 172.21.15.37:0/561843960 learned_addr learned my addr 172.21.15.37:0/561843960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08259c640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 -- 172.21.15.37:0/561843960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 msgr2=0x55cf452ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 -- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 msgr2=0x55cf452f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08b59e640 1 -- 172.21.15.37:0/561843960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4502a620 con 0x55cf452f1800 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.648+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc08b59e640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf452ee970 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc084009ca0 tx=0x7fc084005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f3ea065f435086 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc0727fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc0727fc640 1 -- 172.21.15.37:0/561843960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08405a060 con 0x55cf452f1800 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc0727fc640 1 -- 172.21.15.37:0/561843960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fc084003b70 con 0x55cf452f1800 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc084036070 con 0x55cf452f1800 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc0727fc640 1 -- 172.21.15.37:0/561843960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc084042070 con 0x55cf452f1800 2024-04-23T18:33:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc093de7740 10 client.16875 fetch_fsmap learned FSMap version 45 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cf452f4430 con 0x55cf452f1800 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc0727fc640 1 -- 172.21.15.37:0/561843960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084036070 con 0x55cf452f1800 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.649+0000 7fc0727fc640 1 -- 172.21.15.37:0/561843960 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7fc0840305c0 con 0x55cf452f1800 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc093de7740 10 client.16875 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc093de7740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc093de7740 20 client.16875 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc093de7740 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf453014c0 0x55cf453038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc093de7740 4 client.16875 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf45259390 con 0x55cf453014c0 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf453014c0 0x55cf453038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf453014c0 0x55cf453038b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e05748bf838bb82 server_cookie=f72808f9ee2e55ec in_seq=0 out_seq=0 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.650+0000 7fc0727fc640 10 client.16875 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc0727fc640 1 -- 172.21.15.37:0/561843960 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf45259390 con 0x55cf453014c0 2024-04-23T18:33:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc0727fc640 10 client.16875 handle_command_reply: tid=0 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 client.16875 shutdown 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 msgr2=0x55cf452ee970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf452ee970 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc084009ca0 tx=0x7fc084005c30 comp rx=0 tx=0).stop 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 shutdown_connections 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf453014c0 0x55cf453038b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf452ee970 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc0727fc640 0 client.16875 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 --2- 172.21.15.37:0/561843960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 >> 172.21.15.37:0/561843960 conn(0x55cf452e2b30 msgr2=0x55cf452e8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 shutdown_connections 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 1 -- 172.21.15.37:0/561843960 wait complete. 2024-04-23T18:33:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.651+0000 7fc093de7740 20 client.16875 trim_cache size 0 max 16384 2024-04-23T18:33:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 Processor -- start 2024-04-23T18:33:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 -- start start 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf45309090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1b04b0 con 0x55cf452f1800 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1add50 con 0x55cf452e6620 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.668+0000 7fc093de7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1abbc0 con 0x55cf452e6f40 2024-04-23T18:33:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf45309090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf45309090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43902/0 (socket says 172.21.15.37:43902) 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 -- 172.21.15.37:0/1280467569 learned_addr learned my addr 172.21.15.37:0/1280467569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08259c640 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 -- 172.21.15.37:0/1280467569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 msgr2=0x55cf452ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 -- 172.21.15.37:0/1280467569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 msgr2=0x55cf452fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 -- 172.21.15.37:0/1280467569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf452ec980 con 0x55cf452f1800 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc08b59e640 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf45309090 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc084030990 tx=0x7fc084009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89945e20b1447fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc080d99640 1 -- 172.21.15.37:0/1280467569 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08405a070 con 0x55cf452f1800 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc080d99640 1 -- 172.21.15.37:0/1280467569 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08404b020 con 0x55cf452f1800 2024-04-23T18:33:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.669+0000 7fc080d99640 1 -- 172.21.15.37:0/1280467569 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08402f660 con 0x55cf452f1800 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 -- 172.21.15.37:0/1280467569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 msgr2=0x55cf45309090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf45309090 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc084030990 tx=0x7fc084009800 comp rx=0 tx=0).stop 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 -- 172.21.15.37:0/1280467569 shutdown_connections 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452f1800 0x55cf45309090 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6f40 0x55cf452ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 --2- 172.21.15.37:0/1280467569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452e6620 0x55cf452fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 -- 172.21.15.37:0/1280467569 >> 172.21.15.37:0/1280467569 conn(0x55cf452e2b30 msgr2=0x55cf452e1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 -- 172.21.15.37:0/1280467569 shutdown_connections 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 -- 172.21.15.37:0/1280467569 wait complete. 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.670+0000 7fc093de7740 1 -- start start 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 Processor -- start 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6620 0x55cf452f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 0x55cf452f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf45365900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1ac210 con 0x55cf452e6f40 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1ad680 con 0x55cf452f1800 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1ac8e0 con 0x55cf452e6620 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc093de7740 10 client.0 _ll_register_callbacks cb 0x7fc084009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 0x55cf452f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 0x55cf452f6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43906/0 (socket says 172.21.15.37:43906) 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 -- 172.21.15.37:0/1027533823 learned_addr learned my addr 172.21.15.37:0/1027533823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6620 0x55cf452f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08b59e640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf45365900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 -- 172.21.15.37:0/1027533823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6620 msgr2=0x55cf452f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6620 0x55cf452f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 -- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 msgr2=0x55cf45365900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf45365900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08259c640 1 -- 172.21.15.37:0/1027533823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf452f73b0 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.671+0000 7fc08b59e640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf45365900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc08259c640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 0x55cf452f6ab0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc07400c6a0 tx=0x7fc074099200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ab04605eaf68d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc0727fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc0727fc640 1 -- 172.21.15.37:0/1027533823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07409c610 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc0727fc640 1 -- 172.21.15.37:0/1027533823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fc0740d5040 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08404b020 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc0727fc640 1 -- 172.21.15.37:0/1027533823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc07409c430 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 10 client.16881 fetch_fsmap learned FSMap version 45 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cf45366fa0 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc0727fc640 1 -- 172.21.15.37:0/1027533823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08404b020 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc0727fc640 1 -- 172.21.15.37:0/1027533823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7fc0740ce500 con 0x55cf452e6f40 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 10 client.16881 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 20 client.16881 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf45367700 0x55cf45369b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 4 client.16881 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.672+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf45253200 con 0x55cf45367700 2024-04-23T18:33:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.673+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf45367700 0x55cf45369b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.673+0000 7fc08ad9d640 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf45367700 0x55cf45369b20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27bde84fab02c4f server_cookie=a0607a4d3a7b72f4 in_seq=0 out_seq=0 2024-04-23T18:33:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.673+0000 7fc0727fc640 10 client.16881 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc0727fc640 1 -- 172.21.15.37:0/1027533823 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf45253200 con 0x55cf45367700 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc0727fc640 10 client.16881 handle_command_reply: tid=0 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 client.16881 shutdown 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 msgr2=0x55cf452f6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 0x55cf452f6ab0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc07400c6a0 tx=0x7fc074099200 comp rx=0 tx=0).stop 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 shutdown_connections 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55cf45367700 0x55cf45369b20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf452f1800 0x55cf45365900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc0727fc640 0 client.16881 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf452e6f40 0x55cf452f6ab0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 --2- 172.21.15.37:0/1027533823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cf452e6620 0x55cf452f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.714+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 >> 172.21.15.37:0/1027533823 conn(0x55cf452e2b30 msgr2=0x55cf452e1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 shutdown_connections 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc093de7740 1 -- 172.21.15.37:0/1027533823 wait complete. 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc093de7740 20 client.16881 trim_cache size 0 max 16384 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:33:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:33:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0780a3dc0 msgr2=0x7fc0780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0780a3dc0 0x7fc0780a6280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc08404d000 tx=0x7fc08404e000 comp rx=0 tx=0).stop 2024-04-23T18:33:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 msgr2=0x7fc08c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 0x7fc08c198bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc07400e9f0 tx=0x7fc07400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 shutdown_connections 2024-04-23T18:33:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc0780a3dc0 0x7fc0780a6280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc08c19cf20 0x7fc08c19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc08c106170 0x7fc08c198bf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 --2- 172.21.15.37:0/2885614983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc08c1010f0 0x7fc08c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 >> 172.21.15.37:0/2885614983 conn(0x7fc08c077fd0 msgr2=0x7fc08c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 shutdown_connections 2024-04-23T18:33:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:12.715+0000 7fc0910cf640 1 -- 172.21.15.37:0/2885614983 wait complete. 2024-04-23T18:33:12.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 620353 3.71 MB/sec execute 557 sec latency 2193.313 ms 2024-04-23T18:33:13.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 622103 3.73 MB/sec execute 558 sec latency 14.305 ms 2024-04-23T18:33:14.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:14 smithi172 ceph-mon[32456]: pgmap v401: 97 pgs: 97 active+clean; 479 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2024-04-23T18:33:14.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:14 smithi173 ceph-mon[32906]: pgmap v401: 97 pgs: 97 active+clean; 479 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2024-04-23T18:33:14.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:14 smithi037 ceph-mon[29404]: pgmap v401: 97 pgs: 97 active+clean; 479 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2024-04-23T18:33:14.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 623219 3.73 MB/sec execute 559 sec latency 345.348 ms 2024-04-23T18:33:15.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 623219 3.72 MB/sec execute 560 sec latency 1345.443 ms 2024-04-23T18:33:16.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:16 smithi172 ceph-mon[32456]: pgmap v402: 97 pgs: 97 active+clean; 479 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-04-23T18:33:16.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:16 smithi173 ceph-mon[32906]: pgmap v402: 97 pgs: 97 active+clean; 479 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-04-23T18:33:16.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:16 smithi037 ceph-mon[29404]: pgmap v402: 97 pgs: 97 active+clean; 479 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-04-23T18:33:16.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 623219 3.72 MB/sec execute 561 sec latency 2345.565 ms 2024-04-23T18:33:17.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 623219 3.71 MB/sec execute 562 sec latency 3345.681 ms 2024-04-23T18:33:18.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:18 smithi172 ceph-mon[32456]: pgmap v403: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:18.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:18 smithi173 ceph-mon[32906]: pgmap v403: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:18.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:18 smithi037 ceph-mon[29404]: pgmap v403: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:18.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 623968 3.71 MB/sec execute 563 sec latency 4055.828 ms 2024-04-23T18:33:19.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 625669 3.72 MB/sec execute 564 sec latency 14.612 ms 2024-04-23T18:33:20.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:20 smithi172 ceph-mon[32456]: pgmap v404: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:20.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:20 smithi173 ceph-mon[32906]: pgmap v404: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:20.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:20 smithi037 ceph-mon[29404]: pgmap v404: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:20.856 INFO:tasks.workunit.client.0.smithi037.stdout: 1 626768 3.72 MB/sec execute 565 sec latency 388.013 ms 2024-04-23T18:33:21.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 626768 3.72 MB/sec execute 566 sec latency 1388.111 ms 2024-04-23T18:33:22.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:22 smithi172 ceph-mon[32456]: pgmap v405: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:22.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:22 smithi173 ceph-mon[32906]: pgmap v405: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:22.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:22 smithi037 ceph-mon[29404]: pgmap v405: 97 pgs: 97 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-04-23T18:33:22.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 627494 3.71 MB/sec execute 567 sec latency 2115.974 ms 2024-04-23T18:33:23.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 629193 3.73 MB/sec execute 568 sec latency 19.903 ms 2024-04-23T18:33:24.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:24 smithi172 ceph-mon[32456]: pgmap v406: 97 pgs: 97 active+clean; 482 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-04-23T18:33:24.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:24 smithi173 ceph-mon[32906]: pgmap v406: 97 pgs: 97 active+clean; 482 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-04-23T18:33:24.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:24 smithi037 ceph-mon[29404]: pgmap v406: 97 pgs: 97 active+clean; 482 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-04-23T18:33:24.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 630309 3.73 MB/sec execute 569 sec latency 341.923 ms 2024-04-23T18:33:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 -- 172.21.15.37:0/2123621881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 msgr2=0x7fefa410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 --2- 172.21.15.37:0/2123621881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa410ea00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009ca0 tx=0x7fef9c02a950 comp rx=0 tx=0).stop 2024-04-23T18:33:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 -- 172.21.15.37:0/2123621881 shutdown_connections 2024-04-23T18:33:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 --2- 172.21.15.37:0/2123621881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa410ea00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 --2- 172.21.15.37:0/2123621881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa4103b20 0x7fefa410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 --2- 172.21.15.37:0/2123621881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa4103200 0x7fefa41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.814+0000 7fefa942a640 1 -- 172.21.15.37:0/2123621881 >> 172.21.15.37:0/2123621881 conn(0x7fefa40771d0 msgr2=0x7fefa40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.815+0000 7fefa942a640 1 -- 172.21.15.37:0/2123621881 shutdown_connections 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.815+0000 7fefa942a640 1 -- 172.21.15.37:0/2123621881 wait complete. 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.815+0000 7fefa942a640 1 Processor -- start 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.815+0000 7fefa942a640 1 -- start start 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa942a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa4103200 0x7fefa41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa942a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa4103b20 0x7fefa4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa942a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa942a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefa4112240 con 0x7fefa410c5d0 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa942a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41120c0 con 0x7fefa4103200 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa942a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41123c0 con 0x7fefa4103b20 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37070/0 (socket says 172.21.15.37:37070) 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 -- 172.21.15.37:0/2222544645 learned_addr learned my addr 172.21.15.37:0/2222544645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa4103200 0x7fefa41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa27fc640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa4103b20 0x7fefa4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 -- 172.21.15.37:0/2222544645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa4103b20 msgr2=0x7fefa4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa4103b20 0x7fefa4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 -- 172.21.15.37:0/2222544645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa4103200 msgr2=0x7fefa41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa4103200 0x7fefa41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.816+0000 7fefa37fe640 1 -- 172.21.15.37:0/2222544645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa419d640 con 0x7fefa410c5d0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.817+0000 7fefa37fe640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa419cf10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005e30 tx=0x7fef9c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.817+0000 7fef8bfff640 1 -- 172.21.15.37:0/2222544645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c034040 con 0x7fefa410c5d0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.817+0000 7fef8bfff640 1 -- 172.21.15.37:0/2222544645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c030850 con 0x7fefa410c5d0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.817+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa419d8d0 con 0x7fefa410c5d0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.817+0000 7fef8bfff640 1 -- 172.21.15.37:0/2222544645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0046f0 con 0x7fefa410c5d0 2024-04-23T18:33:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.817+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa41a51c0 con 0x7fefa410c5d0 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.819+0000 7fef8bfff640 1 -- 172.21.15.37:0/2222544645 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef9c037020 con 0x7fefa410c5d0 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.819+0000 7fefac21c740 1 Processor -- start 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 -- start start 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 0x55a730747940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a7307566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730759000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a7306ac970 con 0x55a730747560 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7306494a0 con 0x55a730756c10 2024-04-23T18:33:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefac21c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7306a8930 con 0x55a730747e80 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fef8bfff640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fef8c0a3dc0 0x7fef8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 0x55a730747940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 0x55a730747940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37086/0 (socket says 172.21.15.37:37086) 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 -- 172.21.15.37:0/1149191737 learned_addr learned my addr 172.21.15.37:0/1149191737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa37fe640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fef8c0a3dc0 0x7fef8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a7307566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730759000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fef8bfff640 1 -- 172.21.15.37:0/2222544645 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fef9c0f1690 con 0x7fefa410c5d0 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 -- 172.21.15.37:0/1149191737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 msgr2=0x55a7307566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a7307566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 -- 172.21.15.37:0/1149191737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 msgr2=0x55a730759000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730759000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.820+0000 7fefa27fc640 1 -- 172.21.15.37:0/1149191737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7303bf210 con 0x55a730747560 2024-04-23T18:33:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fefa37fe640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fef8c0a3dc0 0x7fef8c0a6280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0c5de0 tx=0x7fef9c02a950 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fefa27fc640 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 0x55a730747940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef98012540 tx=0x7fef9800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edb3e545c470c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fef897fa640 1 -- 172.21.15.37:0/1149191737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98004030 con 0x55a730747560 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fef897fa640 1 -- 172.21.15.37:0/1149191737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980041d0 con 0x55a730747560 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fef897fa640 1 -- 172.21.15.37:0/1149191737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9800f6e0 con 0x55a730747560 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fefac21c740 1 -- 172.21.15.37:0/1149191737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 msgr2=0x55a730747940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.821+0000 7fefac21c740 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 0x55a730747940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef98012540 tx=0x7fef9800b7c0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- 172.21.15.37:0/1149191737 shutdown_connections 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730759000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a7307566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 --2- 172.21.15.37:0/1149191737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747560 0x55a730747940 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- 172.21.15.37:0/1149191737 >> 172.21.15.37:0/1149191737 conn(0x55a730743b30 msgr2=0x55a730743f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- 172.21.15.37:0/1149191737 shutdown_connections 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- 172.21.15.37:0/1149191737 wait complete. 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- start start 2024-04-23T18:33:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 Processor -- start 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a730756540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 0x55a730753830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730753d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a7307597e0 con 0x55a730747e80 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7306470b0 con 0x55a730756c10 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a730759960 con 0x55a730747560 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefac21c740 10 client.0 _ll_register_callbacks cb 0x55a730634d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefa2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 0x55a730753830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefa2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 0x55a730753830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37102/0 (socket says 172.21.15.37:37102) 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.822+0000 7fefa2ffd640 1 -- 172.21.15.37:0/4078730711 learned_addr learned my addr 172.21.15.37:0/4078730711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa27fc640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a730756540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa27fc640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730753d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa2ffd640 1 -- 172.21.15.37:0/4078730711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 msgr2=0x55a730756540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a730756540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa2ffd640 1 -- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 msgr2=0x55a730753d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730753d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa2ffd640 1 -- 172.21.15.37:0/4078730711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73048b620 con 0x55a730747e80 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa27fc640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a730756540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 0x55a730753830 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef9400ba60 tx=0x7fef9400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f8b84d54386d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fef86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fef86ff5640 1 -- 172.21.15.37:0/4078730711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94005020 con 0x55a730747e80 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fef86ff5640 1 -- 172.21.15.37:0/4078730711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fef940051c0 con 0x55a730747e80 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef980041d0 con 0x55a730747e80 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fef86ff5640 1 -- 172.21.15.37:0/4078730711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef94005400 con 0x55a730747e80 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefac21c740 10 client.16893 fetch_fsmap learned FSMap version 45 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.823+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a730765800 con 0x55a730747e80 2024-04-23T18:33:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fef86ff5640 1 -- 172.21.15.37:0/4078730711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980041d0 con 0x55a730747e80 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fef86ff5640 1 -- 172.21.15.37:0/4078730711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7fef9401b6d0 con 0x55a730747e80 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefac21c740 10 client.16893 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefac21c740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefac21c740 20 client.16893 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefac21c740 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a730766d30 0x55a730769140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefac21c740 4 client.16893 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7306ba390 con 0x55a730766d30 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefa27fc640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a730766d30 0x55a730769140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fefa27fc640 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a730766d30 0x55a730769140 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32298137bd4b513 server_cookie=66609ecf09338618 in_seq=0 out_seq=0 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.824+0000 7fef86ff5640 10 client.16893 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fef86ff5640 1 -- 172.21.15.37:0/4078730711 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7306ba390 con 0x55a730766d30 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fef86ff5640 10 client.16893 handle_command_reply: tid=0 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 client.16893 shutdown 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 msgr2=0x55a730753830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 0x55a730753830 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef9400ba60 tx=0x7fef9400bf20 comp rx=0 tx=0).stop 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 shutdown_connections 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a730766d30 0x55a730769140 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730756c10 0x55a730753d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730747e80 0x55a730753830 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fef86ff5640 0 client.16893 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 --2- 172.21.15.37:0/4078730711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a730756540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.825+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 >> 172.21.15.37:0/4078730711 conn(0x55a730743b30 msgr2=0x55a7307498b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.826+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 shutdown_connections 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.826+0000 7fefac21c740 1 -- 172.21.15.37:0/4078730711 wait complete. 2024-04-23T18:33:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.826+0000 7fefac21c740 20 client.16893 trim_cache size 0 max 16384 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 Processor -- start 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 -- start start 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747560 0x55a730762710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a730756070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a73076eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41ae240 con 0x55a730756c10 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41abae0 con 0x55a730747560 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefac21c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41a9970 con 0x55a730747e80 2024-04-23T18:33:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a73076eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a73076eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37118/0 (socket says 172.21.15.37:37118) 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 -- 172.21.15.37:0/3646109683 learned_addr learned my addr 172.21.15.37:0/3646109683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a730756070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747560 0x55a730762710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 -- 172.21.15.37:0/3646109683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 msgr2=0x55a730756070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a730756070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 -- 172.21.15.37:0/3646109683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747560 msgr2=0x55a730762710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747560 0x55a730762710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.841+0000 7fefa27fc640 1 -- 172.21.15.37:0/3646109683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7307540d0 con 0x55a730756c10 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefa27fc640 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a73076eaa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fef9800bfd0 tx=0x7fef9800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f250fe67fd9c8cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fef88ff9640 1 -- 172.21.15.37:0/3646109683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98027040 con 0x55a730756c10 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fef88ff9640 1 -- 172.21.15.37:0/3646109683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980035f0 con 0x55a730756c10 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fef88ff9640 1 -- 172.21.15.37:0/3646109683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9802d5c0 con 0x55a730756c10 2024-04-23T18:33:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 -- 172.21.15.37:0/3646109683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 msgr2=0x55a73076eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a73076eaa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fef9800bfd0 tx=0x7fef9800b390 comp rx=0 tx=0).stop 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 -- 172.21.15.37:0/3646109683 shutdown_connections 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a73076eaa0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747e80 0x55a730756070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 --2- 172.21.15.37:0/3646109683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747560 0x55a730762710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 -- 172.21.15.37:0/3646109683 >> 172.21.15.37:0/3646109683 conn(0x55a730743b30 msgr2=0x55a730742cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 -- 172.21.15.37:0/3646109683 shutdown_connections 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.842+0000 7fefac21c740 1 -- 172.21.15.37:0/3646109683 wait complete. 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 -- start start 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 Processor -- start 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a73075f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747e80 0x55a7307503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a730750b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41a9fc0 con 0x55a730756c10 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41ab410 con 0x55a730747e80 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41aa670 con 0x55a730747560 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefac21c740 10 client.0 _ll_register_callbacks cb 0x7fef9400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a730750b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a730750b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37134/0 (socket says 172.21.15.37:37134) 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 -- 172.21.15.37:0/961139875 learned_addr learned my addr 172.21.15.37:0/961139875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a73075f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747e80 0x55a7307503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 -- 172.21.15.37:0/961139875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 msgr2=0x55a73075f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a73075f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 -- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747e80 msgr2=0x55a7307503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.843+0000 7fefa27fc640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747e80 0x55a7307503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefa27fc640 1 -- 172.21.15.37:0/961139875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7307cc310 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefa27fc640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a730750b00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fef98001eb0 tx=0x7fef98014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66d1832eeeed63d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fef86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fef86ff5640 1 -- 172.21.15.37:0/961139875 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98003860 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fef86ff5640 1 -- 172.21.15.37:0/961139875 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fef98003a00 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef980035f0 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fef86ff5640 1 -- 172.21.15.37:0/961139875 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef98040050 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 10 client.16899 fetch_fsmap learned FSMap version 45 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a7307cd480 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fef86ff5640 1 -- 172.21.15.37:0/961139875 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980035f0 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fef86ff5640 1 -- 172.21.15.37:0/961139875 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8342+0+0 (secure 0 0 0) 0x7fef9802d040 con 0x55a730756c10 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 10 client.16899 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 20 client.16899 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7307cd930 0x55a7307cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 4 client.16899 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.844+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7306b4200 con 0x55a7307cd930 2024-04-23T18:33:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.845+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7307cd930 0x55a7307cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.845+0000 7fef86ff5640 10 client.16899 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.845+0000 7fefa2ffd640 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7307cd930 0x55a7307cfd50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120015ec3f3eaffb server_cookie=d7b9410ea7dae91b in_seq=0 out_seq=0 2024-04-23T18:33:25.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 630309 3.72 MB/sec execute 570 sec latency 1341.971 ms 2024-04-23T18:33:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.890+0000 7fef86ff5640 1 -- 172.21.15.37:0/961139875 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7306b4200 con 0x55a7307cd930 2024-04-23T18:33:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fef86ff5640 10 client.16899 handle_command_reply: tid=0 2024-04-23T18:33:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 client.16899 shutdown 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 msgr2=0x55a730750b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a730750b00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fef98001eb0 tx=0x7fef98014040 comp rx=0 tx=0).stop 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 shutdown_connections 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55a7307cd930 0x55a7307cfd50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a730756c10 0x55a730750b00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a730747e80 0x55a7307503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 --2- 172.21.15.37:0/961139875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a730747560 0x55a73075f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fef86ff5640 0 client.16899 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 >> 172.21.15.37:0/961139875 conn(0x55a730743b30 msgr2=0x55a730742420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 shutdown_connections 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 1 -- 172.21.15.37:0/961139875 wait complete. 2024-04-23T18:33:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.891+0000 7fefac21c740 20 client.16899 trim_cache size 0 max 16384 2024-04-23T18:33:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:33:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:33:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:33:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fef8c0a3dc0 msgr2=0x7fef8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fef8c0a3dc0 0x7fef8c0a6280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0c5de0 tx=0x7fef9c02a950 comp rx=0 tx=0).stop 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 msgr2=0x7fefa419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa419cf10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005e30 tx=0x7fef9c003f10 comp rx=0 tx=0).stop 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 shutdown_connections 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fef8c0a3dc0 0x7fef8c0a6280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa410c5d0 0x7fefa419cf10 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefa4103b20 0x7fefa4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 --2- 172.21.15.37:0/2222544645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa4103200 0x7fefa41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 >> 172.21.15.37:0/2222544645 conn(0x7fefa40771d0 msgr2=0x7fefa40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 shutdown_connections 2024-04-23T18:33:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:25.892+0000 7fefa942a640 1 -- 172.21.15.37:0/2222544645 wait complete. 2024-04-23T18:33:26.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:26 smithi172 ceph-mon[32456]: pgmap v407: 97 pgs: 97 active+clean; 482 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-04-23T18:33:26.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:26 smithi173 ceph-mon[32906]: pgmap v407: 97 pgs: 97 active+clean; 482 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-04-23T18:33:26.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:26 smithi037 ceph-mon[29404]: pgmap v407: 97 pgs: 97 active+clean; 482 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-04-23T18:33:26.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 632475 3.74 MB/sec execute 571 sec latency 1360.847 ms 2024-04-23T18:33:27.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 633725 3.74 MB/sec execute 572 sec latency 18.758 ms 2024-04-23T18:33:28.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:28 smithi172 ceph-mon[32456]: pgmap v408: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 193 op/s 2024-04-23T18:33:28.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:28 smithi173 ceph-mon[32906]: pgmap v408: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 193 op/s 2024-04-23T18:33:28.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:28 smithi037 ceph-mon[29404]: pgmap v408: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 193 op/s 2024-04-23T18:33:28.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 634634 3.74 MB/sec execute 573 sec latency 623.519 ms 2024-04-23T18:33:29.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 636239 3.76 MB/sec execute 574 sec latency 13.130 ms 2024-04-23T18:33:30.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:30 smithi172 ceph-mon[32456]: pgmap v409: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-04-23T18:33:30.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:30 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:33:30.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:30 smithi173 ceph-mon[32906]: pgmap v409: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-04-23T18:33:30.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:30 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:33:30.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:30 smithi037 ceph-mon[29404]: pgmap v409: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-04-23T18:33:30.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:30 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:33:30.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 637402 3.76 MB/sec execute 575 sec latency 254.843 ms 2024-04-23T18:33:31.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 637402 3.75 MB/sec execute 576 sec latency 1254.926 ms 2024-04-23T18:33:32.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:32 smithi172 ceph-mon[32456]: pgmap v410: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-04-23T18:33:32.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:32 smithi173 ceph-mon[32906]: pgmap v410: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-04-23T18:33:32.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:32 smithi037 ceph-mon[29404]: pgmap v410: 97 pgs: 97 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-04-23T18:33:32.857 INFO:tasks.workunit.client.0.smithi037.stdout: 1 638134 3.75 MB/sec execute 577 sec latency 1983.879 ms 2024-04-23T18:33:33.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 639839 3.76 MB/sec execute 578 sec latency 15.163 ms 2024-04-23T18:33:34.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:34 smithi172 ceph-mon[32456]: pgmap v411: 97 pgs: 97 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 199 op/s 2024-04-23T18:33:34.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:34 smithi173 ceph-mon[32906]: pgmap v411: 97 pgs: 97 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 199 op/s 2024-04-23T18:33:34.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:34 smithi037 ceph-mon[29404]: pgmap v411: 97 pgs: 97 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 199 op/s 2024-04-23T18:33:34.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 640948 3.77 MB/sec execute 579 sec latency 345.035 ms 2024-04-23T18:33:35.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 640948 3.76 MB/sec execute 580 sec latency 1345.109 ms 2024-04-23T18:33:36.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:36 smithi172 ceph-mon[32456]: pgmap v412: 97 pgs: 97 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-04-23T18:33:36.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:36 smithi173 ceph-mon[32906]: pgmap v412: 97 pgs: 97 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-04-23T18:33:36.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:36 smithi037 ceph-mon[29404]: pgmap v412: 97 pgs: 97 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-04-23T18:33:36.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 640948 3.75 MB/sec execute 581 sec latency 2345.237 ms 2024-04-23T18:33:37.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 640948 3.75 MB/sec execute 582 sec latency 3345.289 ms 2024-04-23T18:33:38.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:38 smithi172 ceph-mon[32456]: pgmap v413: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-23T18:33:38.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:38 smithi173 ceph-mon[32906]: pgmap v413: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-23T18:33:38.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:38 smithi037 ceph-mon[29404]: pgmap v413: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-23T18:33:38.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 641692 3.74 MB/sec execute 583 sec latency 4052.683 ms 2024-04-23T18:33:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 -- 172.21.15.37:0/3590424209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f349810c580 msgr2=0x7f349810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 --2- 172.21.15.37:0/3590424209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f349810c580 0x7f349810e9d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3490009ca0 tx=0x7f349002a950 comp rx=0 tx=0).stop 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 -- 172.21.15.37:0/3590424209 shutdown_connections 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 --2- 172.21.15.37:0/3590424209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f349810c580 0x7f349810e9d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 --2- 172.21.15.37:0/3590424209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3498103980 0x7f349810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 --2- 172.21.15.37:0/3590424209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3498103060 0x7f3498103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 -- 172.21.15.37:0/3590424209 >> 172.21.15.37:0/3590424209 conn(0x7f3498077fd0 msgr2=0x7f34980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 -- 172.21.15.37:0/3590424209 shutdown_connections 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.989+0000 7f349c9f0640 1 -- 172.21.15.37:0/3590424209 wait complete. 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 Processor -- start 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 -- start start 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3498103060 0x7f3498198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 0x7f3498198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f349810c580 0x7f349819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34981121b0 con 0x7f3498103980 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3498112030 con 0x7f3498103060 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.990+0000 7f349c9f0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3498112330 con 0x7f349810c580 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 0x7f3498198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 0x7f3498198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43168/0 (socket says 172.21.15.37:43168) 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 -- 172.21.15.37:0/2509564779 learned_addr learned my addr 172.21.15.37:0/2509564779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f349659c640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3498103060 0x7f3498198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f3496d9d640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f349810c580 0x7f349819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 -- 172.21.15.37:0/2509564779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f349810c580 msgr2=0x7f349819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f349810c580 0x7f349819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 -- 172.21.15.37:0/2509564779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3498103060 msgr2=0x7f3498198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3498103060 0x7f3498198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 -- 172.21.15.37:0/2509564779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349819d680 con 0x7f3498103980 2024-04-23T18:33:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348ffff640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 0x7f3498198bd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f348000ea60 tx=0x7f348000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348f7fe640 1 -- 172.21.15.37:0/2509564779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348000cde0 con 0x7f3498103980 2024-04-23T18:33:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348f7fe640 1 -- 172.21.15.37:0/2509564779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34800035f0 con 0x7f3498103980 2024-04-23T18:33:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349819d970 con 0x7f3498103980 2024-04-23T18:33:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f348f7fe640 1 -- 172.21.15.37:0/2509564779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480010690 con 0x7f3498103980 2024-04-23T18:33:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.991+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34981a5200 con 0x7f3498103980 2024-04-23T18:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.994+0000 7f348f7fe640 1 -- 172.21.15.37:0/2509564779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34800037a0 con 0x7f3498103980 2024-04-23T18:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.994+0000 7f349f7de740 1 Processor -- start 2024-04-23T18:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 -- start start 2024-04-23T18:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b123970 con 0x55bd1b1bef40 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b0c04a0 con 0x55bd1b1c9800 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349f7de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b11f930 con 0x55bd1b1be620 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f348f7fe640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f34840a3dc0 0x7f34840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43176/0 (socket says 172.21.15.37:43176) 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f3496d9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 -- 172.21.15.37:0/744839061 learned_addr learned my addr 172.21.15.37:0/744839061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f348ffff640 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f3496d9d640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f34840a3dc0 0x7f34840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f348f7fe640 1 -- 172.21.15.37:0/2509564779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f34800d0b50 con 0x7f3498103980 2024-04-23T18:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 -- 172.21.15.37:0/744839061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 msgr2=0x55bd1b1bea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1bea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 -- 172.21.15.37:0/744839061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 msgr2=0x55bd1b1cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.995+0000 7f349659c640 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f349659c640 1 -- 172.21.15.37:0/744839061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1ae36210 con 0x55bd1b1bef40 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f348ffff640 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f3496d9d640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f34840a3dc0 0x7f34840a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3490005fa0 tx=0x7f34900095c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f349659c640 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c92c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3488009d20 tx=0x7f3488011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4432947998cdef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f348cff9640 1 -- 172.21.15.37:0/744839061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348801e070 con 0x55bd1b1bef40 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f348cff9640 1 -- 172.21.15.37:0/744839061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3488019ae0 con 0x55bd1b1bef40 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.996+0000 7f348cff9640 1 -- 172.21.15.37:0/744839061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348800ea80 con 0x55bd1b1bef40 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 -- 172.21.15.37:0/744839061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 msgr2=0x55bd1b1c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c92c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3488009d20 tx=0x7f3488011810 comp rx=0 tx=0).stop 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 -- 172.21.15.37:0/744839061 shutdown_connections 2024-04-23T18:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c92c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 --2- 172.21.15.37:0/744839061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 -- 172.21.15.37:0/744839061 >> 172.21.15.37:0/744839061 conn(0x55bd1b1bab30 msgr2=0x55bd1b1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 -- 172.21.15.37:0/744839061 shutdown_connections 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 -- 172.21.15.37:0/744839061 wait complete. 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 -- start start 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 Processor -- start 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.997+0000 7f349f7de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b1cc430 con 0x55bd1b1bef40 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b0be0b0 con 0x55bd1b1c9800 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b1cc5b0 con 0x55bd1b1be620 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 10 client.0 _ll_register_callbacks cb 0x55bd1b0abd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43180/0 (socket says 172.21.15.37:43180) 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 -- 172.21.15.37:0/245452628 learned_addr learned my addr 172.21.15.37:0/245452628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f3496d9d640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349659c640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 -- 172.21.15.37:0/245452628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 msgr2=0x55bd1b1c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 -- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 msgr2=0x55bd1b1c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 -- 172.21.15.37:0/245452628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1af02620 con 0x55bd1b1bef40 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f3496d9d640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f348ffff640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c6450 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f348000d010 tx=0x7f348006b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c619144998a4d56f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f347dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f347dffb640 1 -- 172.21.15.37:0/245452628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348009d1b0 con 0x55bd1b1bef40 2024-04-23T18:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f347dffb640 1 -- 172.21.15.37:0/245452628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f348006e110 con 0x55bd1b1bef40 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3488019ae0 con 0x55bd1b1bef40 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f347dffb640 1 -- 172.21.15.37:0/245452628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f34800675c0 con 0x55bd1b1bef40 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.998+0000 7f349f7de740 10 client.16911 fetch_fsmap learned FSMap version 46 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bd1b1d8360 con 0x55bd1b1bef40 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f347dffb640 1 -- 172.21.15.37:0/245452628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3488019ae0 con 0x55bd1b1bef40 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f347dffb640 1 -- 172.21.15.37:0/245452628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) ==== 8338+0+0 (secure 0 0 0) 0x7f34800ce4e0 con 0x55bd1b1bef40 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 10 client.16911 fetch_fsmap finished waiting for FSMap version 46 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 20 client.16911 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1d9820 0x55bd1b1dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 4 client.16911 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd1b131390 con 0x55bd1b1d9820 2024-04-23T18:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:38.999+0000 7f349659c640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1d9820 0x55bd1b1dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349659c640 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1d9820 0x55bd1b1dbc10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46679c65defc74a7 server_cookie=7c0e18ea12d4fac8 in_seq=0 out_seq=0 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f347dffb640 10 client.16911 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f347dffb640 1 -- 172.21.15.37:0/245452628 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd1b131390 con 0x55bd1b1d9820 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f347dffb640 10 client.16911 handle_command_reply: tid=0 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 client.16911 shutdown 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 msgr2=0x55bd1b1c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c6450 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f348000d010 tx=0x7f348006b460 comp rx=0 tx=0).stop 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 shutdown_connections 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1d9820 0x55bd1b1dbc10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1c9800 0x55bd1b1c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bef40 0x55bd1b1c6450 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f347dffb640 0 client.16911 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 --2- 172.21.15.37:0/245452628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1be620 0x55bd1b1c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.000+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 >> 172.21.15.37:0/245452628 conn(0x55bd1b1bab30 msgr2=0x55bd1b1c0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.001+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 shutdown_connections 2024-04-23T18:33:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.001+0000 7f349f7de740 1 -- 172.21.15.37:0/245452628 wait complete. 2024-04-23T18:33:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.001+0000 7f349f7de740 20 client.16911 trim_cache size 0 max 16384 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 Processor -- start 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 -- start start 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 0x55bd1b1b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b04f0 con 0x55bd1b1be620 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34981addc0 con 0x55bd1b1bab30 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.017+0000 7f349f7de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f34981abc30 con 0x55bd1b1bef40 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f349659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 0x55bd1b1b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f349659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:55914/0 (socket says 172.21.15.37:55914) 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f349659c640 1 -- 172.21.15.37:0/3333639791 learned_addr learned my addr 172.21.15.37:0/3333639791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 0x55bd1b1b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43188/0 (socket says 172.21.15.37:43188) 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f3496d9d640 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 -- 172.21.15.37:0/3333639791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 msgr2=0x55bd1b1e13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1e13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 -- 172.21.15.37:0/3333639791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 msgr2=0x55bd1b1d52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 -- 172.21.15.37:0/3333639791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1b1c6d20 con 0x55bd1b1be620 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f349659c640 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f348ffff640 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 0x55bd1b1b9b30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f348009ad60 tx=0x7f348000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da19ce83c026ae5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f347ffff640 1 -- 172.21.15.37:0/3333639791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480098d50 con 0x55bd1b1be620 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f347ffff640 1 -- 172.21.15.37:0/3333639791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f348009d1b0 con 0x55bd1b1be620 2024-04-23T18:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.018+0000 7f347ffff640 1 -- 172.21.15.37:0/3333639791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348009c410 con 0x55bd1b1be620 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 -- 172.21.15.37:0/3333639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 msgr2=0x55bd1b1b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 0x55bd1b1b9b30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f348009ad60 tx=0x7f348000ef90 comp rx=0 tx=0).stop 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 -- 172.21.15.37:0/3333639791 shutdown_connections 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1e13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1be620 0x55bd1b1b9b30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 --2- 172.21.15.37:0/3333639791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 -- 172.21.15.37:0/3333639791 >> 172.21.15.37:0/3333639791 conn(0x55bd1b1dbf80 msgr2=0x55bd1b1b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 -- 172.21.15.37:0/3333639791 shutdown_connections 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 -- 172.21.15.37:0/3333639791 wait complete. 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.019+0000 7f349f7de740 1 -- start start 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 Processor -- start 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1be620 0x55bd1b1ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34981ac280 con 0x55bd1b1bab30 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34981ad6f0 con 0x55bd1b1be620 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f34981ac950 con 0x55bd1b1bef40 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349f7de740 10 client.0 _ll_register_callbacks cb 0x7f348009c970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43194/0 (socket says 172.21.15.37:43194) 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 -- 172.21.15.37:0/582082053 learned_addr learned my addr 172.21.15.37:0/582082053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f348ffff640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1be620 0x55bd1b1ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f3496d9d640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 -- 172.21.15.37:0/582082053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 msgr2=0x55bd1b1cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 -- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1be620 msgr2=0x55bd1b1ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1be620 0x55bd1b1ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 -- 172.21.15.37:0/582082053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1b1cf8f0 con 0x55bd1b1bab30 2024-04-23T18:33:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f349659c640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d19e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f348800a010 tx=0x7f3488005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de6cf5940d529e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f347dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.020+0000 7f347dffb640 1 -- 172.21.15.37:0/582082053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348801e040 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f347dffb640 1 -- 172.21.15.37:0/582082053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f34880050b0 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348009d1b0 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f347dffb640 1 -- 172.21.15.37:0/582082053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f34880052b0 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 10 client.16917 fetch_fsmap learned FSMap version 46 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bd1b23fa60 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f347dffb640 1 -- 172.21.15.37:0/582082053 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f348009d1b0 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f347dffb640 1 -- 172.21.15.37:0/582082053 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) ==== 8338+0+0 (secure 0 0 0) 0x7f348800e790 con 0x55bd1b1bab30 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 10 client.16917 fetch_fsmap finished waiting for FSMap version 46 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 20 client.16917 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1c9800 0x55bd1b242000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 4 client.16917 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd1b12b200 con 0x55bd1b1c9800 2024-04-23T18:33:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.021+0000 7f348ffff640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1c9800 0x55bd1b242000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.022+0000 7f347dffb640 10 client.16917 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.022+0000 7f348ffff640 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1c9800 0x55bd1b242000 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87a56e04dd476c1 server_cookie=efd9f33cdc7c4ecc in_seq=0 out_seq=0 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f347dffb640 1 -- 172.21.15.37:0/582082053 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd1b12b200 con 0x55bd1b1c9800 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f347dffb640 10 client.16917 handle_command_reply: tid=0 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 client.16917 shutdown 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 msgr2=0x55bd1b1d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d19e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f348800a010 tx=0x7f3488005db0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 shutdown_connections 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55bd1b1c9800 0x55bd1b242000 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bd1b1bef40 0x55bd1b1cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f347dffb640 0 client.16917 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bd1b1be620 0x55bd1b1ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 --2- 172.21.15.37:0/582082053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bd1b1bab30 0x55bd1b1d19e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 >> 172.21.15.37:0/582082053 conn(0x55bd1b1dbf80 msgr2=0x55bd1b1b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 shutdown_connections 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 1 -- 172.21.15.37:0/582082053 wait complete. 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.059+0000 7f349f7de740 20 client.16917 trim_cache size 0 max 16384 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:33:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f34840a3dc0 msgr2=0x7f34840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f34840a3dc0 0x7f34840a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3490005fa0 tx=0x7f34900095c0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 msgr2=0x7f3498198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 0x7f3498198bd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f348000ea60 tx=0x7f348000ef20 comp rx=0 tx=0).stop 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 shutdown_connections 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f34840a3dc0 0x7f34840a6280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f349810c580 0x7f349819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3498103980 0x7f3498198bd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 --2- 172.21.15.37:0/2509564779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3498103060 0x7f3498198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 >> 172.21.15.37:0/2509564779 conn(0x7f3498077fd0 msgr2=0x7f34980fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 shutdown_connections 2024-04-23T18:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:39.060+0000 7f349c9f0640 1 -- 172.21.15.37:0/2509564779 wait complete. 2024-04-23T18:33:39.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 643334 3.76 MB/sec execute 584 sec latency 14.663 ms 2024-04-23T18:33:40.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:40 smithi172 ceph-mon[32456]: pgmap v414: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-23T18:33:40.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:40 smithi173 ceph-mon[32906]: pgmap v414: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-23T18:33:40.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:40 smithi037 ceph-mon[29404]: pgmap v414: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-23T18:33:40.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 644492 3.76 MB/sec execute 585 sec latency 267.230 ms 2024-04-23T18:33:41.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 644492 3.75 MB/sec execute 586 sec latency 1267.345 ms 2024-04-23T18:33:42.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:42 smithi172 ceph-mon[32456]: pgmap v415: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-23T18:33:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:42 smithi173 ceph-mon[32906]: pgmap v415: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-23T18:33:42.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:42 smithi037 ceph-mon[29404]: pgmap v415: 97 pgs: 97 active+clean; 503 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-23T18:33:42.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 645202 3.75 MB/sec execute 587 sec latency 1999.502 ms 2024-04-23T18:33:43.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 646811 3.76 MB/sec execute 588 sec latency 13.581 ms 2024-04-23T18:33:44.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:44 smithi172 ceph-mon[32456]: pgmap v416: 97 pgs: 97 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 179 op/s 2024-04-23T18:33:44.353 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:44 smithi173 ceph-mon[32906]: pgmap v416: 97 pgs: 97 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 179 op/s 2024-04-23T18:33:44.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:44 smithi037 ceph-mon[29404]: pgmap v416: 97 pgs: 97 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 179 op/s 2024-04-23T18:33:44.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 648042 3.77 MB/sec execute 589 sec latency 201.609 ms 2024-04-23T18:33:45.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 648042 3.76 MB/sec execute 590 sec latency 1201.716 ms 2024-04-23T18:33:46.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:46 smithi172 ceph-mon[32456]: pgmap v417: 97 pgs: 97 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:46.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:46 smithi173 ceph-mon[32906]: pgmap v417: 97 pgs: 97 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:46.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:46 smithi037 ceph-mon[29404]: pgmap v417: 97 pgs: 97 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:46.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 648042 3.75 MB/sec execute 591 sec latency 2201.830 ms 2024-04-23T18:33:47.858 INFO:tasks.workunit.client.0.smithi037.stdout: 1 648042 3.75 MB/sec execute 592 sec latency 3201.880 ms 2024-04-23T18:33:48.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:48 smithi172 ceph-mon[32456]: pgmap v418: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-23T18:33:48.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:48 smithi173 ceph-mon[32906]: pgmap v418: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-23T18:33:48.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:48 smithi037 ceph-mon[29404]: pgmap v418: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-04-23T18:33:48.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 648804 3.74 MB/sec execute 593 sec latency 3902.594 ms 2024-04-23T18:33:49.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 650446 3.76 MB/sec execute 594 sec latency 13.153 ms 2024-04-23T18:33:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:50 smithi172 ceph-mon[32456]: pgmap v419: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:50.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:50 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:50 smithi173 ceph-mon[32906]: pgmap v419: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:50.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:50 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:50 smithi037 ceph-mon[29404]: pgmap v419: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:50.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:50 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:50.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 651583 3.76 MB/sec execute 595 sec latency 290.008 ms 2024-04-23T18:33:51.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:51 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:51.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:51 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:51.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:51 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:33:51.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:51 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:51.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:51 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:51.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:51 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:33:51.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:51 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:51.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:51 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:51.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:51 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:33:51.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 651583 3.75 MB/sec execute 596 sec latency 1290.088 ms 2024-04-23T18:33:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 -- 172.21.15.37:0/3723336548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1061b0 msgr2=0x7f028c10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3723336548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1061b0 0x7f028c10dd80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0284009ca0 tx=0x7f028402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 -- 172.21.15.37:0/3723336548 shutdown_connections 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3723336548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1061b0 0x7f028c10dd80 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3723336548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c101a90 0x7f028c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3723336548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028c1010e0 0x7f028c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 -- 172.21.15.37:0/3723336548 >> 172.21.15.37:0/3723336548 conn(0x7f028c0771d0 msgr2=0x7f028c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 -- 172.21.15.37:0/3723336548 shutdown_connections 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.160+0000 7f0291ba2640 1 -- 172.21.15.37:0/3723336548 wait complete. 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 Processor -- start 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 -- start start 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 0x7f028c1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028c1061b0 0x7f028c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c19a670 0x7f028c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f028c111120 con 0x7f028c1010e0 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f028c110fa0 con 0x7f028c19a670 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f0291ba2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1112a0 con 0x7f028c1061b0 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 0x7f028c1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 0x7f028c1a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42062/0 (socket says 172.21.15.37:42062) 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f028b7fe640 1 -- 172.21.15.37:0/3704982320 learned_addr learned my addr 172.21.15.37:0/3704982320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.161+0000 7f028affd640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028c1061b0 0x7f028c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028bfff640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c19a670 0x7f028c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028b7fe640 1 -- 172.21.15.37:0/3704982320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028c1061b0 msgr2=0x7f028c1a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028b7fe640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028c1061b0 0x7f028c1a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028b7fe640 1 -- 172.21.15.37:0/3704982320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c19a670 msgr2=0x7f028c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028b7fe640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c19a670 0x7f028c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028b7fe640 1 -- 172.21.15.37:0/3704982320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c19b270 con 0x7f028c1010e0 2024-04-23T18:33:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028bfff640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c19a670 0x7f028c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f028b7fe640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 0x7f028c1a05b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f027c00c970 tx=0x7f027c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f0288ff9640 1 -- 172.21.15.37:0/3704982320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c007bb0 con 0x7f028c1010e0 2024-04-23T18:33:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f0288ff9640 1 -- 172.21.15.37:0/3704982320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c007d50 con 0x7f028c1010e0 2024-04-23T18:33:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c1a7350 con 0x7f028c1010e0 2024-04-23T18:33:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f0288ff9640 1 -- 172.21.15.37:0/3704982320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00f450 con 0x7f028c1010e0 2024-04-23T18:33:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.162+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a7750 con 0x7f028c1010e0 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.165+0000 7f0288ff9640 1 -- 172.21.15.37:0/3704982320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f027c004f20 con 0x7f028c1010e0 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.165+0000 7f0294997740 1 Processor -- start 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.165+0000 7f0294997740 1 -- start start 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.165+0000 7f0294997740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c28ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0294997740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c2972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0294997740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c299bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0294997740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c29c1f1970 con 0x55c29c28cf40 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0294997740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c29c18e4a0 con 0x55c29c28c620 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0294997740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c29c1ed930 con 0x55c29c297800 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c2972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0288ff9640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02740a3e40 0x7f02740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c2972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42066/0 (socket says 172.21.15.37:42066) 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 -- 172.21.15.37:0/2146950838 learned_addr learned my addr 172.21.15.37:0/2146950838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028bfff640 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c28ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028bfff640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02740a3e40 0x7f02740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028affd640 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c299bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f0288ff9640 1 -- 172.21.15.37:0/3704982320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f027c0d0590 con 0x7f028c1010e0 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 -- 172.21.15.37:0/2146950838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 msgr2=0x55c29c299bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c299bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 -- 172.21.15.37:0/2146950838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 msgr2=0x55c29c28ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c28ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028b7fe640 1 -- 172.21.15.37:0/2146950838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29bf04210 con 0x55c29c28cf40 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.166+0000 7f028bfff640 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c28ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f028bfff640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02740a3e40 0x7f02740a6300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0284004af0 tx=0x7f02840095c0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f028b7fe640 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c2972c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f027c0cdc40 tx=0x7f027c004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5766b73fd37818c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0271ffb640 1 -- 172.21.15.37:0/2146950838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c01d070 con 0x55c29c28cf40 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0271ffb640 1 -- 172.21.15.37:0/2146950838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c09cbf0 con 0x55c29c28cf40 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0271ffb640 1 -- 172.21.15.37:0/2146950838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0cdf20 con 0x55c29c28cf40 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 -- 172.21.15.37:0/2146950838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 msgr2=0x55c29c2972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c2972c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f027c0cdc40 tx=0x7f027c004980 comp rx=0 tx=0).stop 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 -- 172.21.15.37:0/2146950838 shutdown_connections 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c299bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c2972c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 --2- 172.21.15.37:0/2146950838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c28ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.167+0000 7f0294997740 1 -- 172.21.15.37:0/2146950838 >> 172.21.15.37:0/2146950838 conn(0x55c29c288b30 msgr2=0x55c29c288f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 -- 172.21.15.37:0/2146950838 shutdown_connections 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 -- 172.21.15.37:0/2146950838 wait complete. 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 -- start start 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 Processor -- start 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c297120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c28cf40 0x55c29c294410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c297800 0x55c29c294950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c29c29a3d0 con 0x55c29c28c620 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c29c18c0b0 con 0x55c29c297800 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.168+0000 7f0294997740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c29c29a550 con 0x55c29c28cf40 2024-04-23T18:33:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f0294997740 10 client.0 _ll_register_callbacks cb 0x55c29c179d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c297120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c297120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42070/0 (socket says 172.21.15.37:42070) 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 -- 172.21.15.37:0/895652347 learned_addr learned my addr 172.21.15.37:0/895652347 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028affd640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c28cf40 0x55c29c294410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028bfff640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c297800 0x55c29c294950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 -- 172.21.15.37:0/895652347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c28cf40 msgr2=0x55c29c294410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c28cf40 0x55c29c294410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 -- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c297800 msgr2=0x55c29c294950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c297800 0x55c29c294950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 -- 172.21.15.37:0/895652347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29bfd0620 con 0x55c29c28c620 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028bfff640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c297800 0x55c29c294950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f028b7fe640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c297120 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f027c0cdd50 tx=0x7f027c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fc85d7dd7d9843 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f026affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f026affd640 1 -- 172.21.15.37:0/895652347 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c01d070 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f026affd640 1 -- 172.21.15.37:0/895652347 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f027c005210 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f027c09cbf0 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f026affd640 1 -- 172.21.15.37:0/895652347 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f027c0a5160 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f0294997740 10 client.16929 fetch_fsmap learned FSMap version 46 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.169+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c29c2a62b0 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f026affd640 1 -- 172.21.15.37:0/895652347 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c09cbf0 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f026affd640 1 -- 172.21.15.37:0/895652347 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) ==== 8338+0+0 (secure 0 0 0) 0x7f027c0bb1f0 con 0x55c29c28c620 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f0294997740 10 client.16929 fetch_fsmap finished waiting for FSMap version 46 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f0294997740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f0294997740 20 client.16929 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f0294997740 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c2a7600 0x55c29c2a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f0294997740 4 client.16929 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c29c1ff390 con 0x55c29c2a7600 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.170+0000 7f028affd640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c2a7600 0x55c29c2a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.171+0000 7f028affd640 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c2a7600 0x55c29c2a99f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9941b21aebc586 server_cookie=48f5054c788aff0c in_seq=0 out_seq=0 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.171+0000 7f026affd640 10 client.16929 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.171+0000 7f026affd640 1 -- 172.21.15.37:0/895652347 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c29c1ff390 con 0x55c29c2a7600 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.171+0000 7f026affd640 10 client.16929 handle_command_reply: tid=0 2024-04-23T18:33:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.171+0000 7f0294997740 1 client.16929 shutdown 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 msgr2=0x55c29c297120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c297120 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f027c0cdd50 tx=0x7f027c0048f0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 shutdown_connections 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c2a7600 0x55c29c2a99f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c297800 0x55c29c294950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c28cf40 0x55c29c294410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 --2- 172.21.15.37:0/895652347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c297120 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f026affd640 0 client.16929 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 >> 172.21.15.37:0/895652347 conn(0x55c29c288b30 msgr2=0x55c29c28e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 shutdown_connections 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 1 -- 172.21.15.37:0/895652347 wait complete. 2024-04-23T18:33:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.172+0000 7f0294997740 20 client.16929 trim_cache size 0 max 16384 2024-04-23T18:33:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 Processor -- start 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 -- start start 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c2a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c293ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c2af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1b2630 con 0x55c29c28cf40 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1aff00 con 0x55c29c28c620 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f0294997740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1add70 con 0x55c29c297800 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f028affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c293ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f028affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c293ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42074/0 (socket says 172.21.15.37:42074) 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f028affd640 1 -- 172.21.15.37:0/1971420398 learned_addr learned my addr 172.21.15.37:0/1971420398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.187+0000 7f028bfff640 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c2af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028b7fe640 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c2a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028affd640 1 -- 172.21.15.37:0/1971420398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 msgr2=0x55c29c2af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028affd640 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c2af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028affd640 1 -- 172.21.15.37:0/1971420398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 msgr2=0x55c29c2a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028affd640 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c2a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028affd640 1 -- 172.21.15.37:0/1971420398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29c2a3dc0 con 0x55c29c28cf40 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028b7fe640 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c2a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f028affd640 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c293ca0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f028000d800 tx=0x7f028000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dbb950495ae0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f02717fa640 1 -- 172.21.15.37:0/1971420398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280003de0 con 0x55c29c28cf40 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f02717fa640 1 -- 172.21.15.37:0/1971420398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028001d8e0 con 0x55c29c28cf40 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.188+0000 7f02717fa640 1 -- 172.21.15.37:0/1971420398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280011d70 con 0x55c29c28cf40 2024-04-23T18:33:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 -- 172.21.15.37:0/1971420398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 msgr2=0x55c29c293ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c293ca0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f028000d800 tx=0x7f028000dda0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 -- 172.21.15.37:0/1971420398 shutdown_connections 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c2af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28cf40 0x55c29c293ca0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 --2- 172.21.15.37:0/1971420398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28c620 0x55c29c2a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 -- 172.21.15.37:0/1971420398 >> 172.21.15.37:0/1971420398 conn(0x55c29c288b30 msgr2=0x55c29c287cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 -- 172.21.15.37:0/1971420398 shutdown_connections 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 -- 172.21.15.37:0/1971420398 wait complete. 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 -- start start 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.189+0000 7f0294997740 1 Processor -- start 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c29d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28cf40 0x55c29c29d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c29dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1ae3c0 con 0x55c29c28c620 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1af830 con 0x55c29c28cf40 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1aea90 con 0x55c29c297800 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 10 client.0 _ll_register_callbacks cb 0x7f027c007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c29d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c29d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42082/0 (socket says 172.21.15.37:42082) 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 -- 172.21.15.37:0/3701002064 learned_addr learned my addr 172.21.15.37:0/3701002064 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028bfff640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c29dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028affd640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28cf40 0x55c29c29d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 -- 172.21.15.37:0/3701002064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 msgr2=0x55c29c29dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c29dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 -- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28cf40 msgr2=0x55c29c29d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28cf40 0x55c29c29d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 -- 172.21.15.37:0/3701002064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29c30ca00 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028bfff640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c29dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f028b7fe640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c29d360 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f027c09c480 tx=0x7f027c09aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6c1c17b3a281e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f026affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f026affd640 1 -- 172.21.15.37:0/3701002064 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c01d070 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f026affd640 1 -- 172.21.15.37:0/3701002064 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f027c0a4cd0 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f028001d8e0 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f026affd640 1 -- 172.21.15.37:0/3701002064 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f027c09c2c0 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 10 client.16935 fetch_fsmap learned FSMap version 46 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.190+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c29c30dc60 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f026affd640 1 -- 172.21.15.37:0/3701002064 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028001d8e0 con 0x55c29c28c620 2024-04-23T18:33:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f026affd640 1 -- 172.21.15.37:0/3701002064 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) ==== 8338+0+0 (secure 0 0 0) 0x7f027c098240 con 0x55c29c28c620 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f0294997740 10 client.16935 fetch_fsmap finished waiting for FSMap version 46 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f0294997740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f0294997740 20 client.16935 populate_metadata read hostname 'smithi037' 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f0294997740 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c30e220 0x55c29c310640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f0294997740 4 client.16935 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c29c1f9200 con 0x55c29c30e220 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f028affd640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c30e220 0x55c29c310640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f026affd640 10 client.16935 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.191+0000 7f028affd640 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c30e220 0x55c29c310640 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fac0c443202fb71 server_cookie=6980325518d1c008 in_seq=0 out_seq=0 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f026affd640 1 -- 172.21.15.37:0/3701002064 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c29c1f9200 con 0x55c29c30e220 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f026affd640 10 client.16935 handle_command_reply: tid=0 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f0294997740 1 client.16935 shutdown 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 msgr2=0x55c29c29d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f0294997740 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c29d360 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f027c09c480 tx=0x7f027c09aa00 comp rx=0 tx=0).stop 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 shutdown_connections 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f0294997740 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x55c29c30e220 0x55c29c310640 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.237+0000 7f0294997740 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c29c297800 0x55c29c29dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0294997740 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c29c28cf40 0x55c29c29d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0294997740 1 --2- 172.21.15.37:0/3701002064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c29c28c620 0x55c29c29d360 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f026affd640 0 client.16935 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 >> 172.21.15.37:0/3701002064 conn(0x55c29c288b30 msgr2=0x55c29c287cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 shutdown_connections 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0294997740 1 -- 172.21.15.37:0/3701002064 wait complete. 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0294997740 20 client.16935 trim_cache size 0 max 16384 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:33:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:33:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02740a3e40 msgr2=0x7f02740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02740a3e40 0x7f02740a6300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0284004af0 tx=0x7f02840095c0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 msgr2=0x7f028c1a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 0x7f028c1a05b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f027c00c970 tx=0x7f027c00ce30 comp rx=0 tx=0).stop 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 shutdown_connections 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f02740a3e40 0x7f02740a6300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c19a670 0x7f028c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f028c1061b0 0x7f028c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 --2- 172.21.15.37:0/3704982320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f028c1010e0 0x7f028c1a05b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 >> 172.21.15.37:0/3704982320 conn(0x7f028c0771d0 msgr2=0x7f028c10cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 shutdown_connections 2024-04-23T18:33:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:33:52.238+0000 7f0291ba2640 1 -- 172.21.15.37:0/3704982320 wait complete. 2024-04-23T18:33:52.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:52 smithi172 ceph-mon[32456]: pgmap v420: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:52.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:52 smithi173 ceph-mon[32906]: pgmap v420: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:52.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:52 smithi037 ceph-mon[29404]: pgmap v420: 97 pgs: 97 active+clean; 513 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-04-23T18:33:52.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 653255 3.76 MB/sec execute 597 sec latency 1653.312 ms 2024-04-23T18:33:53.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 654478 3.77 MB/sec execute 598 sec latency 12.823 ms 2024-04-23T18:33:54.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:54 smithi172 ceph-mon[32456]: pgmap v421: 97 pgs: 97 active+clean; 518 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-04-23T18:33:54.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:54 smithi173 ceph-mon[32906]: pgmap v421: 97 pgs: 97 active+clean; 518 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-04-23T18:33:54.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:54 smithi037 ceph-mon[29404]: pgmap v421: 97 pgs: 97 active+clean; 518 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-04-23T18:33:54.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 655132 3.77 MB/sec execute 599 sec latency 638.546 ms 2024-04-23T18:33:55.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 cleanup 600 sec 2024-04-23T18:33:56.336 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:56 smithi172 ceph-mon[32456]: pgmap v422: 97 pgs: 97 active+clean; 518 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-04-23T18:33:56.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:56 smithi173 ceph-mon[32906]: pgmap v422: 97 pgs: 97 active+clean; 518 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-04-23T18:33:56.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:56 smithi037 ceph-mon[29404]: pgmap v422: 97 pgs: 97 active+clean; 518 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-04-23T18:33:56.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 cleanup 601 sec 2024-04-23T18:33:57.859 INFO:tasks.workunit.client.0.smithi037.stdout: 1 cleanup 602 sec 2024-04-23T18:33:58.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:33:58 smithi173 ceph-mon[32906]: pgmap v423: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-04-23T18:33:58.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:33:58 smithi037 ceph-mon[29404]: pgmap v423: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-04-23T18:33:58.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:33:58 smithi172 ceph-mon[32456]: pgmap v423: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: 0 cleanup 603 sec 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: Operation Count AvgLat MaxLat 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: ---------------------------------------- 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: NTCreateX 71713 2.172 25114.930 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: Close 52657 0.003 0.093 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: Rename 3041 94.983 64757.670 2024-04-23T18:33:58.775 INFO:tasks.workunit.client.0.smithi037.stdout: Unlink 14501 3.674 20961.311 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Deltree 2 109.855 164.745 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Mkdir 1 0.005 0.005 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Qpathinfo 64976 1.358 11661.988 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Qfileinfo 11393 0.002 0.027 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Qfsinfo 11938 0.198 30.421 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Sfileinfo 5844 0.034 4.141 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Find 25139 0.190 27.795 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: WriteX 35797 0.080 26.199 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: ReadX 112512 0.013 5.456 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: LockX 234 0.856 2.130 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: UnlockX 234 0.733 2.188 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: Flush 5031 2.755 39.031 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-23T18:33:58.776 INFO:tasks.workunit.client.0.smithi037.stdout:Throughput 3.7671 MB/sec 1 clients 1 procs max_latency=64757.685 ms 2024-04-23T18:33:58.777 INFO:tasks.workunit.client.0.smithi037.stderr:+ dbench 10 2024-04-23T18:33:58.777 INFO:tasks.workunit.client.0.smithi037.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-23T18:33:58.777 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-23T18:33:58.777 INFO:tasks.workunit.client.0.smithi037.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-23T18:33:58.777 INFO:tasks.workunit.client.0.smithi037.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-23T18:33:58.877 INFO:tasks.workunit.client.0.smithi037.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-23T18:33:58.877 INFO:tasks.workunit.client.0.smithi037.stdout:releasing clients 2024-04-23T18:33:59.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 362 129.93 MB/sec warmup 1 sec latency 58.503 ms 2024-04-23T18:34:00.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:00 smithi173 ceph-mon[32906]: pgmap v424: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:34:00.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:00 smithi037 ceph-mon[29404]: pgmap v424: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:34:00.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:00 smithi172 ceph-mon[32456]: pgmap v424: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:34:00.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 765 104.13 MB/sec warmup 2 sec latency 88.606 ms 2024-04-23T18:34:01.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 1012 74.95 MB/sec warmup 3 sec latency 84.042 ms 2024-04-23T18:34:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:02 smithi173 ceph-mon[32906]: pgmap v425: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:34:02.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:02 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:02.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:02 smithi037 ceph-mon[29404]: pgmap v425: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:34:02.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:02 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:02.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:02 smithi172 ceph-mon[32456]: pgmap v425: 97 pgs: 97 active+clean; 523 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-04-23T18:34:02.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:02 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:02.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 1473 65.11 MB/sec warmup 4 sec latency 112.463 ms 2024-04-23T18:34:03.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 2183 54.24 MB/sec warmup 5 sec latency 156.693 ms 2024-04-23T18:34:04.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:04 smithi173 ceph-mon[32906]: pgmap v426: 97 pgs: 97 active+clean; 535 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-04-23T18:34:04.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:04 smithi037 ceph-mon[29404]: pgmap v426: 97 pgs: 97 active+clean; 535 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-04-23T18:34:04.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:04 smithi172 ceph-mon[32456]: pgmap v426: 97 pgs: 97 active+clean; 535 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-04-23T18:34:04.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 2638 51.70 MB/sec warmup 6 sec latency 1156.750 ms 2024-04-23T18:34:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 -- 172.21.15.37:0/1674019428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 msgr2=0x7f68e4106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 --2- 172.21.15.37:0/1674019428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e4106260 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009ca0 tx=0x7f68dc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 -- 172.21.15.37:0/1674019428 shutdown_connections 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 --2- 172.21.15.37:0/1674019428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68e4101f20 0x7f68e410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 --2- 172.21.15.37:0/1674019428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 --2- 172.21.15.37:0/1674019428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e4106260 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 -- 172.21.15.37:0/1674019428 >> 172.21.15.37:0/1674019428 conn(0x7f68e4077370 msgr2=0x7f68e40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 -- 172.21.15.37:0/1674019428 shutdown_connections 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.331+0000 7f68eaf08640 1 -- 172.21.15.37:0/1674019428 wait complete. 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 Processor -- start 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 -- start start 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68e4101f20 0x7f68e4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41121d0 con 0x7f68e4105e80 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4112050 con 0x7f68e4101560 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68eaf08640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4112350 con 0x7f68e4101f20 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:33900/0 (socket says 172.21.15.37:33900) 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47838/0 (socket says 172.21.15.37:47838) 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 -- 172.21.15.37:0/3599635110 learned_addr learned my addr 172.21.15.37:0/3599635110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e3fff640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68e4101f20 0x7f68e4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 -- 172.21.15.37:0/3599635110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68e4101f20 msgr2=0x7f68e4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68e4101f20 0x7f68e4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 -- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 msgr2=0x7f68e419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.332+0000 7f68e8ca4640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68e8ca4640 1 -- 172.21.15.37:0/3599635110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e419d5c0 con 0x7f68e4101560 2024-04-23T18:34:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68e94a5640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68e8ca4640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41985d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006010 tx=0x7f68dc0316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68e1ffb640 1 -- 172.21.15.37:0/3599635110 <== mon.2 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc003b70 con 0x7f68e4101560 2024-04-23T18:34:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68e1ffb640 1 -- 172.21.15.37:0/3599635110 <== mon.2 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68dc003d10 con 0x7f68e4101560 2024-04-23T18:34:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e419d850 con 0x7f68e4101560 2024-04-23T18:34:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68e1ffb640 1 -- 172.21.15.37:0/3599635110 <== mon.2 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc02fa00 con 0x7f68e4101560 2024-04-23T18:34:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.333+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41a5140 con 0x7f68e4101560 2024-04-23T18:34:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 Processor -- start 2024-04-23T18:34:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e1ffb640 1 -- 172.21.15.37:0/3599635110 <== mon.2 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68dc02fba0 con 0x7f68e4101560 2024-04-23T18:34:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 -- start start 2024-04-23T18:34:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 0x562cf9022a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf902d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562cf8f87970 con 0x562cf9022620 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562cf8f244a0 con 0x562cf9022f40 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68edc25740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562cf8f83930 con 0x562cf902d800 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 0x562cf9022a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e8ca4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf902d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 0x562cf9022a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47848/0 (socket says 172.21.15.37:47848) 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 -- 172.21.15.37:0/1132944522 learned_addr learned my addr 172.21.15.37:0/1132944522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e3fff640 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 -- 172.21.15.37:0/1132944522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 msgr2=0x562cf902fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 -- 172.21.15.37:0/1132944522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 msgr2=0x562cf902d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf902d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e94a5640 1 -- 172.21.15.37:0/1132944522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf8c9a210 con 0x562cf9022620 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e8ca4640 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf902d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.336+0000 7f68e1ffb640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68cc0a3e40 0x7f68cc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.337+0000 7f68e8ca4640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68cc0a3e40 0x7f68cc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.337+0000 7f68e94a5640 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 0x562cf9022a00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f68d4011540 tx=0x7f68d4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27326f5ea3923852 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.337+0000 7f68e1ffb640 1 -- 172.21.15.37:0/3599635110 <== mon.2 v2:172.21.15.172:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f68dc0f1b60 con 0x7f68e4101560 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.337+0000 7f68caffd640 1 -- 172.21.15.37:0/1132944522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d400f5c0 con 0x562cf9022620 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.337+0000 7f68caffd640 1 -- 172.21.15.37:0/1132944522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d400fc10 con 0x562cf9022620 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.337+0000 7f68caffd640 1 -- 172.21.15.37:0/1132944522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4019c30 con 0x562cf9022620 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 -- 172.21.15.37:0/1132944522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 msgr2=0x562cf9022a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68e8ca4640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68cc0a3e40 0x7f68cc0a6300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0ef410 tx=0x7f68dc0f56b0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 0x562cf9022a00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f68d4011540 tx=0x7f68d4011a00 comp rx=0 tx=0).stop 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 -- 172.21.15.37:0/1132944522 shutdown_connections 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf902d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 --2- 172.21.15.37:0/1132944522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022620 0x562cf9022a00 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 -- 172.21.15.37:0/1132944522 >> 172.21.15.37:0/1132944522 conn(0x562cf901eb30 msgr2=0x562cf901ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 -- 172.21.15.37:0/1132944522 shutdown_connections 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 -- 172.21.15.37:0/1132944522 wait complete. 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 -- start start 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.338+0000 7f68edc25740 1 Processor -- start 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022620 0x562cf902d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 0x562cf902a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562cf9030430 con 0x562cf9022f40 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562cf8f220b0 con 0x562cf9022620 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562cf90305b0 con 0x562cf902d800 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 10 client.0 _ll_register_callbacks cb 0x562cf8f0fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 0x562cf902a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022620 0x562cf902d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 0x562cf902a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47856/0 (socket says 172.21.15.37:47856) 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022620 0x562cf902d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.37:33922/0 (socket says 172.21.15.37:33922) 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 -- 172.21.15.37:0/2284272492 learned_addr learned my addr 172.21.15.37:0/2284272492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 -- 172.21.15.37:0/2284272492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 msgr2=0x562cf902a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 -- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022620 msgr2=0x562cf902d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022620 0x562cf902d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 -- 172.21.15.37:0/2284272492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf8d66620 con 0x562cf9022f40 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68e3fff640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 0x562cf902a420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f68d8012700 tx=0x7f68d8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da4acc136030f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:34:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68c3fff640 1 -- 172.21.15.37:0/2284272492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004320 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68c3fff640 1 -- 172.21.15.37:0/2284272492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f68d80119a0 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.339+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d400fc10 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68c3fff640 1 -- 172.21.15.37:0/2284272492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f68d80044c0 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 10 client.16947 fetch_fsmap learned FSMap version 48 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562cf903c380 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68c3fff640 1 -- 172.21.15.37:0/2284272492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d400fc10 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68c3fff640 1 -- 172.21.15.37:0/2284272492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f68d800f440 con 0x562cf9022f40 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 10 client.16947 fetch_fsmap finished waiting for FSMap version 48 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 20 client.16947 populate_metadata read hostname 'smithi037' 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf903d890 0x562cf903fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 4 client.16947 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf8f95390 con 0x562cf903d890 2024-04-23T18:34:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.340+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf903d890 0x562cf903fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.341+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf903d890 0x562cf903fca0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f63a1b2ce8b7b0b6 server_cookie=d00ab1fb1c2e4971 in_seq=0 out_seq=0 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.341+0000 7f68c3fff640 10 client.16947 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.341+0000 7f68c3fff640 1 -- 172.21.15.37:0/2284272492 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562cf8f95390 con 0x562cf903d890 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.341+0000 7f68c3fff640 10 client.16947 handle_command_reply: tid=0 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.341+0000 7f68edc25740 1 client.16947 shutdown 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 msgr2=0x562cf902a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 0x562cf902a420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f68d8012700 tx=0x7f68d8012bc0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 shutdown_connections 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf903d890 0x562cf903fca0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf902d800 0x562cf902a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf9022f40 0x562cf902a420 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 --2- 172.21.15.37:0/2284272492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022620 0x562cf902d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68c3fff640 0 client.16947 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 >> 172.21.15.37:0/2284272492 conn(0x562cf901eb30 msgr2=0x562cf90248b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 shutdown_connections 2024-04-23T18:34:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 1 -- 172.21.15.37:0/2284272492 wait complete. 2024-04-23T18:34:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.342+0000 7f68edc25740 20 client.16947 trim_cache size 0 max 16384 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 Processor -- start 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 -- start start 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf90392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf90455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9045af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41b0560 con 0x562cf902d800 2024-04-23T18:34:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41addd0 con 0x562cf9022f40 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68edc25740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41abc40 con 0x562cf9022620 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.357+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9045af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9045af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47868/0 (socket says 172.21.15.37:47868) 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 -- 172.21.15.37:0/2693092432 learned_addr learned my addr 172.21.15.37:0/2693092432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e3fff640 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf90455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf90392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 -- 172.21.15.37:0/2693092432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 msgr2=0x562cf90392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf90392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 -- 172.21.15.37:0/2693092432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 msgr2=0x562cf90455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf90455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 -- 172.21.15.37:0/2693092432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf901a2e0 con 0x562cf902d800 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68e94a5640 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9045af0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f68d4011f00 tx=0x7f68d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cba3e531bbd2af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68ca7fc640 1 -- 172.21.15.37:0/2693092432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4005190 con 0x562cf902d800 2024-04-23T18:34:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68ca7fc640 1 -- 172.21.15.37:0/2693092432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d4005330 con 0x562cf902d800 2024-04-23T18:34:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.358+0000 7f68ca7fc640 1 -- 172.21.15.37:0/2693092432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d40198f0 con 0x562cf902d800 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 -- 172.21.15.37:0/2693092432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 msgr2=0x562cf9045af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9045af0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f68d4011f00 tx=0x7f68d4005f90 comp rx=0 tx=0).stop 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 -- 172.21.15.37:0/2693092432 shutdown_connections 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9045af0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf90455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 --2- 172.21.15.37:0/2693092432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf90392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 -- 172.21.15.37:0/2693092432 >> 172.21.15.37:0/2693092432 conn(0x562cf901eb30 msgr2=0x562cf901d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 -- 172.21.15.37:0/2693092432 shutdown_connections 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 -- 172.21.15.37:0/2693092432 wait complete. 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.359+0000 7f68edc25740 1 -- start start 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 Processor -- start 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf9035880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf9032c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9033150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41ac290 con 0x562cf902d800 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41ad700 con 0x562cf9022f40 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41ac960 con 0x562cf9022620 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68edc25740 10 client.0 _ll_register_callbacks cb 0x7f68d8009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68e94a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9033150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47876/0 (socket says 172.21.15.37:47876) 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68e94a5640 1 -- 172.21.15.37:0/3110170034 learned_addr learned my addr 172.21.15.37:0/3110170034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68e3fff640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf9032c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.360+0000 7f68e94a5640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf9035880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68e94a5640 1 -- 172.21.15.37:0/3110170034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 msgr2=0x562cf9035880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68e94a5640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf9035880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68e94a5640 1 -- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 msgr2=0x562cf9032c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68e94a5640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf9032c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68e94a5640 1 -- 172.21.15.37:0/3110170034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf9033690 con 0x562cf902d800 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68e94a5640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9033150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f68d4011fd0 tx=0x7f68d40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f1ec350dcf1829 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68c3fff640 1 -- 172.21.15.37:0/3110170034 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d400f5c0 con 0x562cf902d800 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68c3fff640 1 -- 172.21.15.37:0/3110170034 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f68d400f760 con 0x562cf902d800 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d4005330 con 0x562cf902d800 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68c3fff640 1 -- 172.21.15.37:0/3110170034 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f68d400f960 con 0x562cf902d800 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68edc25740 10 client.16953 fetch_fsmap learned FSMap version 48 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.361+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562cf90a7f10 con 0x562cf902d800 2024-04-23T18:34:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68c3fff640 1 -- 172.21.15.37:0/3110170034 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d4005330 con 0x562cf902d800 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68c3fff640 1 -- 172.21.15.37:0/3110170034 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f68d4032020 con 0x562cf902d800 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68edc25740 10 client.16953 fetch_fsmap finished waiting for FSMap version 48 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68edc25740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68edc25740 20 client.16953 populate_metadata read hostname 'smithi037' 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68edc25740 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf90a82e0 0x562cf90aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68edc25740 4 client.16953 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cf902acf0 con 0x562cf90a82e0 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68e3fff640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf90a82e0 0x562cf90aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68c3fff640 10 client.16953 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.362+0000 7f68e3fff640 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf90a82e0 0x562cf90aa6d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204a29b448c7ce73 server_cookie=e3db6fb3a79ae03f in_seq=0 out_seq=0 2024-04-23T18:34:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:05 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:05.360 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:05 smithi173 ceph-mon[32906]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:34:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:05 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:05 smithi037 ceph-mon[29404]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68c3fff640 1 -- 172.21.15.37:0/3110170034 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cf902acf0 con 0x562cf90a82e0 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68c3fff640 10 client.16953 handle_command_reply: tid=0 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 client.16953 shutdown 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 msgr2=0x562cf9033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9033150 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f68d4011fd0 tx=0x7f68d40096a0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 shutdown_connections 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x562cf90a82e0 0x562cf90aa6d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cf902d800 0x562cf9033150 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cf9022f40 0x562cf9032c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68c3fff640 0 client.16953 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 --2- 172.21.15.37:0/3110170034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562cf9022620 0x562cf9035880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 >> 172.21.15.37:0/3110170034 conn(0x562cf901eb30 msgr2=0x562cf901dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 shutdown_connections 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 1 -- 172.21.15.37:0/3110170034 wait complete. 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.427+0000 7f68edc25740 20 client.16953 trim_cache size 0 max 16384 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-23T18:34:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-23T18:34:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-23T18:34:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68cc0a3e40 msgr2=0x7f68cc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68cc0a3e40 0x7f68cc0a6300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0ef410 tx=0x7f68dc0f56b0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 msgr2=0x7f68e41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41985d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006010 tx=0x7f68dc0316f0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 shutdown_connections 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f68cc0a3e40 0x7f68cc0a6300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68e4105e80 0x7f68e419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68e4101f20 0x7f68e4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 --2- 172.21.15.37:0/3599635110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e4101560 0x7f68e41985d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 >> 172.21.15.37:0/3599635110 conn(0x7f68e4077370 msgr2=0x7f68e4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 shutdown_connections 2024-04-23T18:34:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-23T18:34:05.428+0000 7f68eaf08640 1 -- 172.21.15.37:0/3599635110 wait complete. 2024-04-23T18:34:05.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:05 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:05.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:05 smithi172 ceph-mon[32456]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:34:05.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3219 49.99 MB/sec warmup 7 sec latency 1675.085 ms 2024-04-23T18:34:06.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:06 smithi037 ceph-mon[29404]: pgmap v428: 97 pgs: 97 active+clean; 535 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 274 op/s 2024-04-23T18:34:06.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:06 smithi172 ceph-mon[32456]: pgmap v428: 97 pgs: 97 active+clean; 535 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 274 op/s 2024-04-23T18:34:06.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:06 smithi173 ceph-mon[32906]: pgmap v428: 97 pgs: 97 active+clean; 535 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 274 op/s 2024-04-23T18:34:06.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3632 48.09 MB/sec warmup 8 sec latency 105.000 ms 2024-04-23T18:34:07.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3793 43.24 MB/sec warmup 9 sec latency 74.479 ms 2024-04-23T18:34:08.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:08 smithi037 ceph-mon[29404]: pgmap v429: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:08.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:08 smithi172 ceph-mon[32456]: pgmap v429: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:08.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:08 smithi173 ceph-mon[32906]: pgmap v429: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:08.877 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3987 39.43 MB/sec warmup 10 sec latency 85.930 ms 2024-04-23T18:34:09.548 INFO:tasks.workunit.client.0.smithi037.stdout:[4209] write failed on handle 10768 (Resource temporarily unavailable) 2024-04-23T18:34:09.548 INFO:tasks.workunit.client.0.smithi037.stdout:Child failed with status 1 2024-04-23T18:34:09.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:34:09.574 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-23T18:34:09.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T18:34:09.945 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-23T18:34:10.197 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=155226d01bcf40c897eee7a68ed62293 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-23T18:34:10.200 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-23T18:34:10.210 INFO:tasks.quiescer:joining Quiescers 2024-04-23T18:34:10.211 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-23T18:34:10.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:10.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:10 smithi037 ceph-mon[29404]: pgmap v430: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 -- 172.21.15.37:0/1063936751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 msgr2=0x7f13341014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 --2- 172.21.15.37:0/1063936751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341014f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1324009900 tx=0x7f132402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 -- 172.21.15.37:0/1063936751 shutdown_connections 2024-04-23T18:34:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 --2- 172.21.15.37:0/1063936751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 0x7f133410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 --2- 172.21.15.37:0/1063936751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1334101ac0 0x7f1334105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 --2- 172.21.15.37:0/1063936751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341014f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 -- 172.21.15.37:0/1063936751 >> 172.21.15.37:0/1063936751 conn(0x7f1334077fb0 msgr2=0x7f13340783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 -- 172.21.15.37:0/1063936751 shutdown_connections 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 -- 172.21.15.37:0/1063936751 wait complete. 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 Processor -- start 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.331+0000 7f133a951640 1 -- start start 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f133a951640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f133a951640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 0x7f1334199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f133a951640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f133419a0a0 0x7f13341a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f133a951640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13341110f0 con 0x7f1334101110 2024-04-23T18:34:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f133a951640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1334110f70 con 0x7f1334106190 2024-04-23T18:34:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f133a951640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1334111270 con 0x7f133419a0a0 2024-04-23T18:34:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47890/0 (socket says 172.21.15.37:47890) 2024-04-23T18:34:10.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 -- 172.21.15.37:0/2101180947 learned_addr learned my addr 172.21.15.37:0/2101180947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:10.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1338eee640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f133419a0a0 0x7f13341a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f13337fe640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 0x7f1334199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 -- 172.21.15.37:0/2101180947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f133419a0a0 msgr2=0x7f13341a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f133419a0a0 0x7f13341a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 -- 172.21.15.37:0/2101180947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 msgr2=0x7f1334199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 0x7f1334199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 -- 172.21.15.37:0/2101180947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13341a4e20 con 0x7f1334101110 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f13337fe640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 0x7f1334199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.332+0000 7f1333fff640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341995d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1324005940 tx=0x7f13240040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.333+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132403f030 con 0x7f1334101110 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.333+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13240044c0 con 0x7f1334101110 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.333+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13341a50b0 con 0x7f1334101110 2024-04-23T18:34:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.333+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324035070 con 0x7f1334101110 2024-04-23T18:34:10.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.333+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a55a0 con 0x7f1334101110 2024-04-23T18:34:10.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.334+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13340691c0 con 0x7f1334101110 2024-04-23T18:34:10.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.335+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1324042030 con 0x7f1334101110 2024-04-23T18:34:10.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.335+0000 7f13317fa640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f131c0a3d20 0x7f131c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.336+0000 7f13337fe640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f131c0a3d20 0x7f131c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.336+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f13240f0cb0 con 0x7f1334101110 2024-04-23T18:34:10.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.336+0000 7f13337fe640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f131c0a3d20 0x7f131c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f133406b100 tx=0x7f1328005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.340+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13240bd290 con 0x7f1334101110 2024-04-23T18:34:10.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.567+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13341093c0 con 0x7f1334101110 2024-04-23T18:34:10.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.567+0000 7f13317fa640 1 -- 172.21.15.37:0/2101180947 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11877 (secure 0 0 0) 0x7f13240bcb90 con 0x7f1334101110 2024-04-23T18:34:10.559 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:10.560 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:34:03.554284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24484,24509,24479]},"id":1}]} 2024-04-23T18:34:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 48 2024-04-23T18:34:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.569+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f131c0a3d20 msgr2=0x7f131c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.569+0000 7f133a951640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f131c0a3d20 0x7f131c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f133406b100 tx=0x7f1328005f90 comp rx=0 tx=0).stop 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.569+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 msgr2=0x7f13341995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.569+0000 7f133a951640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341995d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1324005940 tx=0x7f13240040c0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 shutdown_connections 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f131c0a3d20 0x7f131c0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f133419a0a0 0x7f13341a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1334106190 0x7f1334199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 --2- 172.21.15.37:0/2101180947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1334101110 0x7f13341995d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 >> 172.21.15.37:0/2101180947 conn(0x7f1334077fb0 msgr2=0x7f13340ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 shutdown_connections 2024-04-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.570+0000 7f133a951640 1 -- 172.21.15.37:0/2101180947 wait complete. 2024-04-23T18:34:10.573 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --cancel --all' 2024-04-23T18:34:10.573 DEBUG:teuthology.orchestra.run.smithi037:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --cancel --all 2024-04-23T18:34:10.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:10 smithi172 ceph-mon[32456]: pgmap v430: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:10.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:10 smithi173 ceph-mon[32906]: pgmap v430: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:10.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 -- 172.21.15.37:0/1296688132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e406b1f0 msgr2=0x7f08e4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 --2- 172.21.15.37:0/1296688132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e406b1f0 0x7f08e4106b90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009900 tx=0x7f08d402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 -- 172.21.15.37:0/1296688132 shutdown_connections 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 --2- 172.21.15.37:0/1296688132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 0x7f08e410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 --2- 172.21.15.37:0/1296688132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e406b1f0 0x7f08e4106b90 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 --2- 172.21.15.37:0/1296688132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e406a840 0x7f08e406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 -- 172.21.15.37:0/1296688132 >> 172.21.15.37:0/1296688132 conn(0x7f08e4077eb0 msgr2=0x7f08e40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 -- 172.21.15.37:0/1296688132 shutdown_connections 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.728+0000 7f08eb730640 1 -- 172.21.15.37:0/1296688132 wait complete. 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 Processor -- start 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 -- start start 2024-04-23T18:34:10.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e406a840 0x7f08e4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 0x7f08e4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 0x7f08e419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4112190 con 0x7f08e419cff0 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4112010 con 0x7f08e4107270 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08eb730640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4112310 con 0x7f08e406a840 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08e9ccd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 0x7f08e419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08e9ccd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 0x7f08e419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47910/0 (socket says 172.21.15.37:47910) 2024-04-23T18:34:10.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08e9ccd640 1 -- 172.21.15.37:0/2170527125 learned_addr learned my addr 172.21.15.37:0/2170527125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08e94cc640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e406a840 0x7f08e4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.729+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 0x7f08e4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e9ccd640 1 -- 172.21.15.37:0/2170527125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e406a840 msgr2=0x7f08e4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e9ccd640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e406a840 0x7f08e4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e9ccd640 1 -- 172.21.15.37:0/2170527125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 msgr2=0x7f08e4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e9ccd640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 0x7f08e4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e9ccd640 1 -- 172.21.15.37:0/2170527125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e419dae0 con 0x7f08e419cff0 2024-04-23T18:34:10.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 0x7f08e4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:10.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08e9ccd640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 0x7f08e419d4a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009fd0 tx=0x7f08dc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08da7fc640 1 -- 172.21.15.37:0/2170527125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc017070 con 0x7f08e419cff0 2024-04-23T18:34:10.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08da7fc640 1 -- 172.21.15.37:0/2170527125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08dc003e20 con 0x7f08e419cff0 2024-04-23T18:34:10.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e419ddd0 con 0x7f08e419cff0 2024-04-23T18:34:10.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08da7fc640 1 -- 172.21.15.37:0/2170527125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc010460 con 0x7f08e419cff0 2024-04-23T18:34:10.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.730+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e41a55c0 con 0x7f08e419cff0 2024-04-23T18:34:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08da7fc640 1 -- 172.21.15.37:0/2170527125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f08dc003970 con 0x7f08e419cff0 2024-04-23T18:34:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 Processor -- start 2024-04-23T18:34:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 -- start start 2024-04-23T18:34:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 0x564af27ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27ac7c0 0x564af27acc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564af2707130 con 0x564af27a1bb0 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564af26a3c50 con 0x564af27a1200 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08ee446740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564af27030f0 con 0x564af27ac7c0 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.733+0000 7f08e94cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 0x564af27ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 0x564af27ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47924/0 (socket says 172.21.15.37:47924) 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 -- 172.21.15.37:0/3557814562 learned_addr learned my addr 172.21.15.37:0/3557814562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08da7fc640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f08d00a3da0 0x7f08d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 -- 172.21.15.37:0/3557814562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27ac7c0 msgr2=0x564af27acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27ac7c0 0x564af27acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 -- 172.21.15.37:0/3557814562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 msgr2=0x564af27a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 -- 172.21.15.37:0/3557814562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af26bdfd0 con 0x564af27a1bb0 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e9ccd640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f08d00a3da0 0x7f08d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08da7fc640 1 -- 172.21.15.37:0/2170527125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f08dc0d0520 con 0x7f08e419cff0 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08e94cc640 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 0x564af27ac1a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f08e000bd10 tx=0x7f08e00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ddc6404e8faed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08cf7fe640 1 -- 172.21.15.37:0/3557814562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004030 con 0x564af27a1bb0 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08cf7fe640 1 -- 172.21.15.37:0/3557814562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e00035f0 con 0x564af27a1bb0 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.734+0000 7f08cf7fe640 1 -- 172.21.15.37:0/3557814562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0011770 con 0x564af27a1bb0 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 -- 172.21.15.37:0/3557814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 msgr2=0x564af27ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 0x564af27ac1a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f08e000bd10 tx=0x7f08e00107a0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08e9ccd640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f08d00a3da0 0x7f08d00a6260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08dc09c160 tx=0x7f08dc06bd40 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 -- 172.21.15.37:0/3557814562 shutdown_connections 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27ac7c0 0x564af27acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27a1bb0 0x564af27ac1a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 --2- 172.21.15.37:0/3557814562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 -- 172.21.15.37:0/3557814562 >> 172.21.15.37:0/3557814562 conn(0x564af279d620 msgr2=0x564af279da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 -- 172.21.15.37:0/3557814562 shutdown_connections 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 -- 172.21.15.37:0/3557814562 wait complete. 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 -- start start 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.735+0000 7f08ee446740 1 Processor -- start 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564af27b3740 con 0x564af27ac7c0 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564af26a1640 con 0x564af27a1bb0 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564af27b38c0 con 0x564af27a1200 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ee446740 10 client.0 _ll_register_callbacks cb 0x564af26a1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47934/0 (socket says 172.21.15.37:47934) 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1877196637 learned_addr learned my addr 172.21.15.37:0/1877196637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e94cc640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1877196637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 msgr2=0x564af27b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 msgr2=0x564af27b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1877196637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af24e5a20 con 0x564af27ac7c0 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b0ba0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f08d4033000 tx=0x7f08d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680dc924a1eb8507 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.736+0000 7f08ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:34:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ccff9640 1 -- 172.21.15.37:0/1877196637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4035040 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ccff9640 1 -- 172.21.15.37:0/1877196637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f08d40035f0 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e00035f0 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ccff9640 1 -- 172.21.15.37:0/1877196637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f08d4050050 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 10 client.16971 fetch_fsmap learned FSMap version 48 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564af25461b0 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ccff9640 1 -- 172.21.15.37:0/1877196637 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e00035f0 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ccff9640 1 -- 172.21.15.37:0/1877196637 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f08d403a050 con 0x564af27ac7c0 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 10 client.16971 fetch_fsmap finished waiting for FSMap version 48 2024-04-23T18:34:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 10 client.16971 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 20 client.16971 populate_metadata read hostname 'smithi037' 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af27c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 4 client.16971 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564af2714b50 con 0x564af27c0be0 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.737+0000 7f08e94cc640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af27c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.738+0000 7f08e94cc640 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af27c3010 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee73a174990f0fe7 server_cookie=3d7c8b06812545bf in_seq=0 out_seq=0 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.738+0000 7f08ccff9640 10 client.16971 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:10.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.738+0000 7f08ccff9640 1 -- 172.21.15.37:0/1877196637 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564af2714b50 con 0x564af27c0be0 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.738+0000 7f08ccff9640 10 client.16971 handle_command_reply: tid=0 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 client.16971 shutdown 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 msgr2=0x564af27b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b0ba0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f08d4033000 tx=0x7f08d4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 shutdown_connections 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af27c3010 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b0ba0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ccff9640 0 client.16971 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 --2- 172.21.15.37:0/1877196637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 >> 172.21.15.37:0/1877196637 conn(0x564af279d620 msgr2=0x564af27a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 shutdown_connections 2024-04-23T18:34:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 1 -- 172.21.15.37:0/1877196637 wait complete. 2024-04-23T18:34:10.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.739+0000 7f08ee446740 20 client.16971 trim_cache size 0 max 16384 2024-04-23T18:34:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 Processor -- start 2024-04-23T18:34:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 -- start start 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27bc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41ae4e0 con 0x564af27ac7c0 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41abd50 con 0x564af27a1bb0 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.756+0000 7f08ee446740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41a9bc0 con 0x564af27a1200 2024-04-23T18:34:10.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47940/0 (socket says 172.21.15.37:47940) 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1102657762 learned_addr learned my addr 172.21.15.37:0/1102657762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1102657762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 msgr2=0x564af27bc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27bc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1102657762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 msgr2=0x564af27adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e94cc640 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 -- 172.21.15.37:0/1102657762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af27a1950 con 0x564af27ac7c0 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e94cc640 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27c88e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x564af2792480 tx=0x7f08d4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628842e3b59f6d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08ceffd640 1 -- 172.21.15.37:0/1102657762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4035070 con 0x564af27ac7c0 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08ceffd640 1 -- 172.21.15.37:0/1102657762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d4031e20 con 0x564af27ac7c0 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.757+0000 7f08ceffd640 1 -- 172.21.15.37:0/1102657762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4005500 con 0x564af27ac7c0 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 -- 172.21.15.37:0/1102657762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 msgr2=0x564af27c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27c88e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x564af2792480 tx=0x7f08d4031400 comp rx=0 tx=0).stop 2024-04-23T18:34:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 -- 172.21.15.37:0/1102657762 shutdown_connections 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27c88e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1bb0 0x564af27adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 --2- 172.21.15.37:0/1102657762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1200 0x564af27bc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 -- 172.21.15.37:0/1102657762 >> 172.21.15.37:0/1102657762 conn(0x564af279d620 msgr2=0x564af27b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 -- 172.21.15.37:0/1102657762 shutdown_connections 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 -- 172.21.15.37:0/1102657762 wait complete. 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 -- start start 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 Processor -- start 2024-04-23T18:34:10.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.758+0000 7f08ee446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1bb0 0x564af27b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41aa210 con 0x564af27ac7c0 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41ab680 con 0x564af27a1200 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41aa8e0 con 0x564af27a1bb0 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 10 client.0 _ll_register_callbacks cb 0x7f08d4031620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47956/0 (socket says 172.21.15.37:47956) 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 -- 172.21.15.37:0/33615901 learned_addr learned my addr 172.21.15.37:0/33615901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 -- 172.21.15.37:0/33615901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1bb0 msgr2=0x564af27b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e94cc640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1bb0 0x564af27b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1bb0 0x564af27b9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 -- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 msgr2=0x564af27b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27b8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 -- 172.21.15.37:0/33615901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af23d9310 con 0x564af27ac7c0 2024-04-23T18:34:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08e8ccb640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b5b20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f08d4031730 tx=0x7f08d4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2e529879abadfb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ccff9640 1 -- 172.21.15.37:0/33615901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d405e040 con 0x564af27ac7c0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ccff9640 1 -- 172.21.15.37:0/33615901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7f08d4031b70 con 0x564af27ac7c0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d4031e20 con 0x564af27ac7c0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ccff9640 1 -- 172.21.15.37:0/33615901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f08d4035040 con 0x564af27ac7c0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 10 client.16977 fetch_fsmap learned FSMap version 48 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.759+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564af28277f0 con 0x564af27ac7c0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ccff9640 1 -- 172.21.15.37:0/33615901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08d4031e20 con 0x564af27ac7c0 2024-04-23T18:34:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ccff9640 1 -- 172.21.15.37:0/33615901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 48) ==== 8346+0+0 (secure 0 0 0) 0x7f08d403a070 con 0x564af27ac7c0 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ee446740 10 client.16977 fetch_fsmap finished waiting for FSMap version 48 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ee446740 10 client.16977 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ee446740 20 client.16977 populate_metadata read hostname 'smithi037' 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ee446740 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af2829eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ee446740 4 client.16977 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 --> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x564af270e9c0 con 0x564af27c0be0 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08e94cc640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af2829eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08e94cc640 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af2829eb0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e4bccfc2c0cf69 server_cookie=6ccd06b45db40586 in_seq=0 out_seq=0 2024-04-23T18:34:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.760+0000 7f08ccff9640 10 client.16977 ms_handle_connect on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ccff9640 1 -- 172.21.15.37:0/33615901 <== mds.0 v2:172.21.15.172:6838/2128854391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x564af270e9c0 con 0x564af27c0be0 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ccff9640 10 client.16977 handle_command_reply: tid=0 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 client.16977 shutdown 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 msgr2=0x564af27b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b5b20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f08d4031730 tx=0x7f08d4009520 comp rx=0 tx=0).stop 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 shutdown_connections 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:6838/2128854391,v1:172.21.15.172:6839/2128854391] conn(0x564af27c0be0 0x564af2829eb0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564af27ac7c0 0x564af27b5b20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ccff9640 0 client.16977 ms_handle_reset on v2:172.21.15.172:6838/2128854391 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564af27a1bb0 0x564af27b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 --2- 172.21.15.37:0/33615901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564af27a1200 0x564af27b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 >> 172.21.15.37:0/33615901 conn(0x564af279d620 msgr2=0x564af279c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 shutdown_connections 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 1 -- 172.21.15.37:0/33615901 wait complete. 2024-04-23T18:34:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08ee446740 20 client.16977 trim_cache size 0 max 16384 2024-04-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 48, 2024-04-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24509, 2024-04-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 26, 2024-04-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": {} 2024-04-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f08d00a3da0 msgr2=0x7f08d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08eb730640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f08d00a3da0 0x7f08d00a6260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08dc09c160 tx=0x7f08dc06bd40 comp rx=0 tx=0).stop 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.761+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 msgr2=0x7f08e419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 0x7f08e419d4a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009fd0 tx=0x7f08dc00b260 comp rx=0 tx=0).stop 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 shutdown_connections 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f08d00a3da0 0x7f08d00a6260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e419cff0 0x7f08e419d4a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08e4107270 0x7f08e4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 --2- 172.21.15.37:0/2170527125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e406a840 0x7f08e4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 >> 172.21.15.37:0/2170527125 conn(0x7f08e4077eb0 msgr2=0x7f08e4108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 shutdown_connections 2024-04-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:10.762+0000 7f08eb730640 1 -- 172.21.15.37:0/2170527125 wait complete. 2024-04-23T18:34:10.767 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 408, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 361, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing root: 110 (ETIMEDOUT) 2024-04-23T18:34:10.771 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-23T18:34:10.782 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:34:10.790 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-23T18:34:10.800 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-23T18:34:10.800 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:34:10.800 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:34:10.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-23T18:34:10.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-23T18:34:10.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:34:11.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 -- 172.21.15.37:0/1997748718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f8101a90 msgr2=0x7f62f8105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 --2- 172.21.15.37:0/1997748718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f8101a90 0x7f62f8105ad0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009900 tx=0x7f62e802a410 comp rx=0 tx=0).stop 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 -- 172.21.15.37:0/1997748718 shutdown_connections 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 --2- 172.21.15.37:0/1997748718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 0x7f62f810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 --2- 172.21.15.37:0/1997748718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f8101a90 0x7f62f8105ad0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 --2- 172.21.15.37:0/1997748718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f81010e0 0x7f62f81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 -- 172.21.15.37:0/1997748718 >> 172.21.15.37:0/1997748718 conn(0x7f62f80771d0 msgr2=0x7f62f80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 -- 172.21.15.37:0/1997748718 shutdown_connections 2024-04-23T18:34:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.076+0000 7f62ff023640 1 -- 172.21.15.37:0/1997748718 wait complete. 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 Processor -- start 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 -- start start 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 0x7f62f8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 0x7f62f8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f819cf50 0x7f62f819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8111140 con 0x7f62f81010e0 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8110fc0 con 0x7f62f819cf50 2024-04-23T18:34:11.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62ff023640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81112c0 con 0x7f62f81061b0 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62fcdbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 0x7f62f8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62fcdbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 0x7f62f8198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47988/0 (socket says 172.21.15.37:47988) 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62fcdbf640 1 -- 172.21.15.37:0/2567257957 learned_addr learned my addr 172.21.15.37:0/2567257957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62fd5c0640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f819cf50 0x7f62f819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.077+0000 7f62effff640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 0x7f62f8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62fcdbf640 1 -- 172.21.15.37:0/2567257957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 msgr2=0x7f62f8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62fcdbf640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 0x7f62f8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62fcdbf640 1 -- 172.21.15.37:0/2567257957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f819cf50 msgr2=0x7f62f819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62fcdbf640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f819cf50 0x7f62f819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62fcdbf640 1 -- 172.21.15.37:0/2567257957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f819da40 con 0x7f62f81010e0 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62effff640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 0x7f62f8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62fcdbf640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 0x7f62f8198690 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d950 tx=0x7f62f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4017030 con 0x7f62f81010e0 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f4004020 con 0x7f62f81010e0 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f819dd30 con 0x7f62f81010e0 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f40047a0 con 0x7f62f81010e0 2024-04-23T18:34:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.078+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a54f0 con 0x7f62f81010e0 2024-04-23T18:34:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.079+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f804fce0 con 0x7f62f81010e0 2024-04-23T18:34:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.080+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62f4009a30 con 0x7f62f81010e0 2024-04-23T18:34:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.081+0000 7f62edffb640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62e00a3dc0 0x7f62e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.082+0000 7f62effff640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62e00a3dc0 0x7f62e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.082+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f62f40d0320 con 0x7f62f81010e0 2024-04-23T18:34:11.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.082+0000 7f62effff640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62e00a3dc0 0x7f62e00a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006010 tx=0x7f62e8000bc0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.087+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62f409c920 con 0x7f62f81010e0 2024-04-23T18:34:11.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.304+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f62f810a460 con 0x7f62e00a3dc0 2024-04-23T18:34:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.313+0000 7f62edffb640 1 -- 172.21.15.37:0/2567257957 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f62e80043b0 con 0x7f62e00a3dc0 2024-04-23T18:34:11.305 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:20:31", "first": "2024-04-23T18:21:00", "last": "2024-04-23T18:34:00", "last_pruned": "2024-04-23T18:34:00", "created_count": 14, "pruned_count": 8, "active": true} 2024-04-23T18:34:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62e00a3dc0 msgr2=0x7f62e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62e00a3dc0 0x7f62e00a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006010 tx=0x7f62e8000bc0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 msgr2=0x7f62f8198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 0x7f62f8198690 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d950 tx=0x7f62f400de10 comp rx=0 tx=0).stop 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 shutdown_connections 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62e00a3dc0 0x7f62e00a6280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f819cf50 0x7f62f819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f81061b0 0x7f62f8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 --2- 172.21.15.37:0/2567257957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f81010e0 0x7f62f8198690 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 >> 172.21.15.37:0/2567257957 conn(0x7f62f80771d0 msgr2=0x7f62f806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 shutdown_connections 2024-04-23T18:34:11.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.315+0000 7f62ff023640 1 -- 172.21.15.37:0/2567257957 wait complete. 2024-04-23T18:34:11.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:34:11.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:11 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2101180947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:11.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:11 smithi037 ceph-mon[29404]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 -- 172.21.15.37:0/803689294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c1011a0 msgr2=0x7f1f4c101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 --2- 172.21.15.37:0/803689294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c1011a0 0x7f1f4c101620 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009900 tx=0x7f1f3c02a400 comp rx=0 tx=0).stop 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 -- 172.21.15.37:0/803689294 shutdown_connections 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 --2- 172.21.15.37:0/803689294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f4c101b60 0x7f1f4c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 --2- 172.21.15.37:0/803689294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c1011a0 0x7f1f4c101620 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 --2- 172.21.15.37:0/803689294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 -- 172.21.15.37:0/803689294 >> 172.21.15.37:0/803689294 conn(0x7f1f4c076c50 msgr2=0x7f1f4c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 -- 172.21.15.37:0/803689294 shutdown_connections 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 -- 172.21.15.37:0/803689294 wait complete. 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 Processor -- start 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.431+0000 7f1f535f7640 1 -- start start 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f535f7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f4c1011a0 0x7f1f4c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f535f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f4c101b60 0x7f1f4c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f535f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f535f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c111060 con 0x7f1f4c106fb0 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f535f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c110ee0 con 0x7f1f4c101b60 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f535f7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c1111e0 con 0x7f1f4c1011a0 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c1a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47998/0 (socket says 172.21.15.37:47998) 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 -- 172.21.15.37:0/3662561815 learned_addr learned my addr 172.21.15.37:0/3662561815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f50b92640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f4c101b60 0x7f1f4c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51393640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f4c1011a0 0x7f1f4c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 -- 172.21.15.37:0/3662561815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f4c1011a0 msgr2=0x7f1f4c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f4c1011a0 0x7f1f4c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 -- 172.21.15.37:0/3662561815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f4c101b60 msgr2=0x7f1f4c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f4c101b60 0x7f1f4c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 -- 172.21.15.37:0/3662561815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c1a4b00 con 0x7f1f4c106fb0 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f51b94640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c1a45c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400cad0 tx=0x7f1f4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.432+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400ae20 con 0x7f1f4c106fb0 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.433+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f44005020 con 0x7f1f4c106fb0 2024-04-23T18:34:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.433+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c1a4cd0 con 0x7f1f4c106fb0 2024-04-23T18:34:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.433+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400f6b0 con 0x7f1f4c106fb0 2024-04-23T18:34:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.433+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c1a5140 con 0x7f1f4c106fb0 2024-04-23T18:34:11.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.433+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c102550 con 0x7f1f4c106fb0 2024-04-23T18:34:11.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.435+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f440051d0 con 0x7f1f4c106fb0 2024-04-23T18:34:11.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.435+0000 7f1f427fc640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1f380a3dc0 0x7f1f380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.435+0000 7f1f51393640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1f380a3dc0 0x7f1f380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.435+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f1f440d0bf0 con 0x7f1f4c106fb0 2024-04-23T18:34:11.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.436+0000 7f1f51393640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1f380a3dc0 0x7f1f380a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f480058c0 tx=0x7f1f4800a2d0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.439+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f44014030 con 0x7f1f4c106fb0 2024-04-23T18:34:11.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:11 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2101180947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:11.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:11 smithi172 ceph-mon[32456]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:34:11.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:11 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2101180947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:11.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:11 smithi173 ceph-mon[32906]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:34:11.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.648+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 --> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1f4c062390 con 0x7f1f380a3dc0 2024-04-23T18:34:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.658+0000 7f1f427fc640 1 -- 172.21.15.37:0/3662561815 <== mgr.15270 v2:172.21.15.37:6800/816900776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1f4c062390 con 0x7f1f380a3dc0 2024-04-23T18:34:11.649 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-23T18:34:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1f380a3dc0 msgr2=0x7f1f380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1f380a3dc0 0x7f1f380a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f480058c0 tx=0x7f1f4800a2d0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 msgr2=0x7f1f4c1a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c1a45c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400cad0 tx=0x7f1f4400cf90 comp rx=0 tx=0).stop 2024-04-23T18:34:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 shutdown_connections 2024-04-23T18:34:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1f380a3dc0 0x7f1f380a6280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f4c106fb0 0x7f1f4c1a45c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f4c101b60 0x7f1f4c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 --2- 172.21.15.37:0/3662561815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f4c1011a0 0x7f1f4c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 >> 172.21.15.37:0/3662561815 conn(0x7f1f4c076c50 msgr2=0x7f1f4c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 shutdown_connections 2024-04-23T18:34:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.660+0000 7f1f535f7640 1 -- 172.21.15.37:0/3662561815 wait complete. 2024-04-23T18:34:11.663 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-23T18:34:11.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:34:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 -- 172.21.15.37:0/3609836103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c41016d0 msgr2=0x7fd2c410e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/3609836103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c41016d0 0x7fd2c410e420 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009900 tx=0x7fd2b402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 -- 172.21.15.37:0/3609836103 shutdown_connections 2024-04-23T18:34:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/3609836103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c410e960 0x7fd2c4110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/3609836103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c41016d0 0x7fd2c410e420 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/3609836103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c4100db0 0x7fd2c4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 -- 172.21.15.37:0/3609836103 >> 172.21.15.37:0/3609836103 conn(0x7fd2c4077fb0 msgr2=0x7fd2c40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 -- 172.21.15.37:0/3609836103 shutdown_connections 2024-04-23T18:34:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.785+0000 7fd2ca88a640 1 -- 172.21.15.37:0/3609836103 wait complete. 2024-04-23T18:34:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 Processor -- start 2024-04-23T18:34:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 -- start start 2024-04-23T18:34:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 0x7fd2c41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c41016d0 0x7fd2c41a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c410e960 0x7fd2c419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4114460 con 0x7fd2c4100db0 2024-04-23T18:34:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41142e0 con 0x7fd2c41016d0 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2ca88a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41145e0 con 0x7fd2c410e960 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 0x7fd2c41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 0x7fd2c41a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48018/0 (socket says 172.21.15.37:48018) 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2458753286 learned_addr learned my addr 172.21.15.37:0/2458753286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.786+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c41016d0 0x7fd2c41a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c8e27640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c410e960 0x7fd2c419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2458753286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c410e960 msgr2=0x7fd2c419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c410e960 0x7fd2c419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2458753286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c41016d0 msgr2=0x7fd2c41a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c41016d0 0x7fd2c41a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2458753286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c419b000 con 0x7fd2c4100db0 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c8e27640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c410e960 0x7fd2c419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 0x7fd2c41a06d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c890 tx=0x7fd2ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac004220 con 0x7fd2c4100db0 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ac0043c0 con 0x7fd2c4100db0 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c419b2f0 con 0x7fd2c4100db0 2024-04-23T18:34:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00f450 con 0x7fd2c4100db0 2024-04-23T18:34:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.787+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41a7590 con 0x7fd2c4100db0 2024-04-23T18:34:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.788+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c40691c0 con 0x7fd2c4100db0 2024-04-23T18:34:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.789+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2ac0035f0 con 0x7fd2c4100db0 2024-04-23T18:34:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.790+0000 7fd2c17fa640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd2b00a3d20 0x7fd2b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.790+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd2b00a3d20 0x7fd2b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.790+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fd2ac0d02f0 con 0x7fd2c4100db0 2024-04-23T18:34:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.790+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd2b00a3d20 0x7fd2b00a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005ab0 tx=0x7fd2b4005a20 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:11.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.795+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2ac014030 con 0x7fd2c4100db0 2024-04-23T18:34:11.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.982+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2c419bdb0 con 0x7fd2c4100db0 2024-04-23T18:34:11.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.984+0000 7fd2c17fa640 1 -- 172.21.15.37:0/2458753286 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+23259 (secure 0 0 0) 0x7fd2ac09c8f0 con 0x7fd2c4100db0 2024-04-23T18:34:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:11.976 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:34:11.037181+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:11.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd2b00a3d20 msgr2=0x7fd2b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd2b00a3d20 0x7fd2b00a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005ab0 tx=0x7fd2b4005a20 comp rx=0 tx=0).stop 2024-04-23T18:34:11.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 msgr2=0x7fd2c41a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:11.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 0x7fd2c41a06d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c890 tx=0x7fd2ac00cd50 comp rx=0 tx=0).stop 2024-04-23T18:34:11.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 shutdown_connections 2024-04-23T18:34:11.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fd2b00a3d20 0x7fd2b00a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c410e960 0x7fd2c419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c41016d0 0x7fd2c41a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 --2- 172.21.15.37:0/2458753286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4100db0 0x7fd2c41a06d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 >> 172.21.15.37:0/2458753286 conn(0x7fd2c4077fb0 msgr2=0x7fd2c410ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 shutdown_connections 2024-04-23T18:34:11.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:11.986+0000 7fd2ca88a640 1 -- 172.21.15.37:0/2458753286 wait complete. 2024-04-23T18:34:11.989 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:34:11.990 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:34:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 -- 172.21.15.37:0/1706956970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 msgr2=0x7f786c10e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 --2- 172.21.15.37:0/1706956970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c10e270 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f785c009900 tx=0x7f785c02a480 comp rx=0 tx=0).stop 2024-04-23T18:34:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 -- 172.21.15.37:0/1706956970 shutdown_connections 2024-04-23T18:34:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 --2- 172.21.15.37:0/1706956970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f786c10e7b0 0x7f786c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 --2- 172.21.15.37:0/1706956970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c10e270 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 --2- 172.21.15.37:0/1706956970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f786c074380 0x7f786c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 -- 172.21.15.37:0/1706956970 >> 172.21.15.37:0/1706956970 conn(0x7f786c0fda60 msgr2=0x7f786c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 -- 172.21.15.37:0/1706956970 shutdown_connections 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 -- 172.21.15.37:0/1706956970 wait complete. 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 Processor -- start 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.104+0000 7f787379b640 1 -- start start 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f787379b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f786c074380 0x7f786c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f787379b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f787379b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f786c10e7b0 0x7f786c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f787379b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f786c1143f0 con 0x7f786c074ca0 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f787379b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f786c114270 con 0x7f786c10e7b0 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f787379b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f786c114570 con 0x7f786c074380 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48030/0 (socket says 172.21.15.37:48030) 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 -- 172.21.15.37:0/1672239123 learned_addr learned my addr 172.21.15.37:0/1672239123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7871537640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f786c074380 0x7f786c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7871d38640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f786c10e7b0 0x7f786c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 -- 172.21.15.37:0/1672239123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f786c074380 msgr2=0x7f786c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f786c074380 0x7f786c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 -- 172.21.15.37:0/1672239123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f786c10e7b0 msgr2=0x7f786c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f786c10e7b0 0x7f786c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 -- 172.21.15.37:0/1672239123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786c19afe0 con 0x7f786c074ca0 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f7870d36640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c1a0be0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f785c005940 tx=0x7f785c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.105+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c03f030 con 0x7f786c074ca0 2024-04-23T18:34:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.106+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c0044c0 con 0x7f786c074ca0 2024-04-23T18:34:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.106+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c035070 con 0x7f786c074ca0 2024-04-23T18:34:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.106+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786c19b270 con 0x7f786c074ca0 2024-04-23T18:34:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.106+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786c1a74f0 con 0x7f786c074ca0 2024-04-23T18:34:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.106+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786c075a30 con 0x7f786c074ca0 2024-04-23T18:34:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.108+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f785c042030 con 0x7f786c074ca0 2024-04-23T18:34:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.108+0000 7f78627fc640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f78580a3d20 0x7f78580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.108+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f785c0f0f10 con 0x7f786c074ca0 2024-04-23T18:34:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.108+0000 7f7871537640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f78580a3d20 0x7f78580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.109+0000 7f7871537640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f78580a3d20 0x7f78580a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7868009b60 tx=0x7f7868005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.112+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f785c0bd570 con 0x7f786c074ca0 2024-04-23T18:34:12.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.302+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f786c109930 con 0x7f786c074ca0 2024-04-23T18:34:12.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.303+0000 7f78627fc640 1 -- 172.21.15.37:0/1672239123 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f785c0bce70 con 0x7f786c074ca0 2024-04-23T18:34:12.295 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.304+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f78580a3d20 msgr2=0x7f78580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.304+0000 7f787379b640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f78580a3d20 0x7f78580a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7868009b60 tx=0x7f7868005f90 comp rx=0 tx=0).stop 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.304+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 msgr2=0x7f786c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.304+0000 7f787379b640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c1a0be0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f785c005940 tx=0x7f785c0040c0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 shutdown_connections 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f78580a3d20 0x7f78580a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f786c10e7b0 0x7f786c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f786c074ca0 0x7f786c1a0be0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 --2- 172.21.15.37:0/1672239123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f786c074380 0x7f786c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 >> 172.21.15.37:0/1672239123 conn(0x7f786c0fda60 msgr2=0x7f786c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 shutdown_connections 2024-04-23T18:34:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.305+0000 7f787379b640 1 -- 172.21.15.37:0/1672239123 wait complete. 2024-04-23T18:34:12.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:12 smithi037 ceph-mon[29404]: pgmap v431: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:12.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:12 smithi037 ceph-mon[29404]: from='client.16983 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:12.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:12 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2458753286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:12.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:34:12.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 -- 172.21.15.37:0/2113619859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394101a40 msgr2=0x7f939410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 --2- 172.21.15.37:0/2113619859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394101a40 0x7f939410dca0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f938c009ca0 tx=0x7f938c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 -- 172.21.15.37:0/2113619859 shutdown_connections 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 --2- 172.21.15.37:0/2113619859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394101a40 0x7f939410dca0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 --2- 172.21.15.37:0/2113619859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93941010a0 0x7f9394101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 --2- 172.21.15.37:0/2113619859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9394105dd0 0x7f93941061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 -- 172.21.15.37:0/2113619859 >> 172.21.15.37:0/2113619859 conn(0x7f9394077c40 msgr2=0x7f9394078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:12.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 -- 172.21.15.37:0/2113619859 shutdown_connections 2024-04-23T18:34:12.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.421+0000 7f939a14f640 1 -- 172.21.15.37:0/2113619859 wait complete. 2024-04-23T18:34:12.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 Processor -- start 2024-04-23T18:34:12.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 -- start start 2024-04-23T18:34:12.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93941010a0 0x7f9394197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9394101a40 0x7f9394197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 0x7f939419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93941111b0 con 0x7f9394105dd0 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9394111030 con 0x7f93941010a0 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f939a14f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9394111330 con 0x7f9394101a40 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f9393fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 0x7f939419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f9393fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 0x7f939419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48050/0 (socket says 172.21.15.37:48050) 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.422+0000 7f9393fff640 1 -- 172.21.15.37:0/2102474959 learned_addr learned my addr 172.21.15.37:0/2102474959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9392ffd640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9394101a40 0x7f9394197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f93937fe640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93941010a0 0x7f9394197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9393fff640 1 -- 172.21.15.37:0/2102474959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9394101a40 msgr2=0x7f9394197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9393fff640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9394101a40 0x7f9394197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9393fff640 1 -- 172.21.15.37:0/2102474959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93941010a0 msgr2=0x7f9394197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9393fff640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93941010a0 0x7f9394197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9393fff640 1 -- 172.21.15.37:0/2102474959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939419c5b0 con 0x7f9394105dd0 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9393fff640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 0x7f939419be80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f938c009430 tx=0x7f938c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c003c30 con 0x7f9394105dd0 2024-04-23T18:34:12.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c003dd0 con 0x7f9394105dd0 2024-04-23T18:34:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939419c840 con 0x7f9394105dd0 2024-04-23T18:34:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c02faa0 con 0x7f9394105dd0 2024-04-23T18:34:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.423+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a4130 con 0x7f9394105dd0 2024-04-23T18:34:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.424+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394102540 con 0x7f9394105dd0 2024-04-23T18:34:12.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.425+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f938c0045f0 con 0x7f9394105dd0 2024-04-23T18:34:12.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.426+0000 7f9390ff9640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f937c0a3d20 0x7f937c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.426+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f938c0f1d10 con 0x7f9394105dd0 2024-04-23T18:34:12.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.426+0000 7f93937fe640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f937c0a3d20 0x7f937c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.426+0000 7f93937fe640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f937c0a3d20 0x7f937c0a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f938800cac0 tx=0x7f938800c4a0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:12.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.430+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f938c0f7020 con 0x7f9394105dd0 2024-04-23T18:34:12.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:12 smithi172 ceph-mon[32456]: pgmap v431: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:12.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:12 smithi172 ceph-mon[32456]: from='client.16983 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:12.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:12 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2458753286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:12.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:12 smithi173 ceph-mon[32906]: pgmap v431: 97 pgs: 97 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 447 op/s 2024-04-23T18:34:12.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:12 smithi173 ceph-mon[32906]: from='client.16983 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:12.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:12 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2458753286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:12.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.619+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93941a4300 con 0x7f9394105dd0 2024-04-23T18:34:12.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.619+0000 7f9390ff9640 1 -- 172.21.15.37:0/2102474959 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f938c0be420 con 0x7f9394105dd0 2024-04-23T18:34:12.611 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f937c0a3d20 msgr2=0x7f937c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f937c0a3d20 0x7f937c0a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f938800cac0 tx=0x7f938800c4a0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 msgr2=0x7f939419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 0x7f939419be80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f938c009430 tx=0x7f938c0319f0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 shutdown_connections 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f937c0a3d20 0x7f937c0a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9394105dd0 0x7f939419be80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9394101a40 0x7f9394197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 --2- 172.21.15.37:0/2102474959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93941010a0 0x7f9394197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.621+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 >> 172.21.15.37:0/2102474959 conn(0x7f9394077c40 msgr2=0x7f93940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:12.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.622+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 shutdown_connections 2024-04-23T18:34:12.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.622+0000 7f939a14f640 1 -- 172.21.15.37:0/2102474959 wait complete. 2024-04-23T18:34:12.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:34:12.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 -- 172.21.15.37:0/799193472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006b1f0 msgr2=0x7fa3a0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 --2- 172.21.15.37:0/799193472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006b1f0 0x7fa3a0106b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa394009900 tx=0x7fa39402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 -- 172.21.15.37:0/799193472 shutdown_connections 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 --2- 172.21.15.37:0/799193472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 0x7fa3a010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 --2- 172.21.15.37:0/799193472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006b1f0 0x7fa3a0106b90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 --2- 172.21.15.37:0/799193472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a006a840 0x7fa3a006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 -- 172.21.15.37:0/799193472 >> 172.21.15.37:0/799193472 conn(0x7fa3a0077eb0 msgr2=0x7fa3a00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:12.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 -- 172.21.15.37:0/799193472 shutdown_connections 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.738+0000 7fa3a6035640 1 -- 172.21.15.37:0/799193472 wait complete. 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 Processor -- start 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 -- start start 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 0x7fa3a0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 0x7fa3a01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a0101900 0x7fa3a01a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01121b0 con 0x7fa3a006a840 2024-04-23T18:34:12.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0112030 con 0x7fa3a0101900 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa3a6035640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0112330 con 0x7fa3a0107270 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 0x7fa3a0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 0x7fa3a0100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48072/0 (socket says 172.21.15.37:48072) 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.739+0000 7fa39f7fe640 1 -- 172.21.15.37:0/849365487 learned_addr learned my addr 172.21.15.37:0/849365487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39effd640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 0x7fa3a01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39ffff640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a0101900 0x7fa3a01a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39f7fe640 1 -- 172.21.15.37:0/849365487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 msgr2=0x7fa3a01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39f7fe640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 0x7fa3a01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39f7fe640 1 -- 172.21.15.37:0/849365487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a0101900 msgr2=0x7fa3a01a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39f7fe640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a0101900 0x7fa3a01a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39f7fe640 1 -- 172.21.15.37:0/849365487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a01a6fc0 con 0x7fa3a006a840 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39effd640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 0x7fa3a01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39ffff640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a0101900 0x7fa3a01a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39f7fe640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 0x7fa3a0100e80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa39000e9f0 tx=0x7fa39000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000cd80 con 0x7fa3a006a840 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3900108e0 con 0x7fa3a006a840 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a01a72b0 con 0x7fa3a006a840 2024-04-23T18:34:12.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390018980 con 0x7fa3a006a840 2024-04-23T18:34:12.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.740+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a7820 con 0x7fa3a006a840 2024-04-23T18:34:12.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.741+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a00691c0 con 0x7fa3a006a840 2024-04-23T18:34:12.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.742+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa390010430 con 0x7fa3a006a840 2024-04-23T18:34:12.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.743+0000 7fa39cff9640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa3880a3dd0 0x7fa3880a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:12.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.743+0000 7fa39effd640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa3880a3dd0 0x7fa3880a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:12.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.743+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fa3900d1110 con 0x7fa3a006a840 2024-04-23T18:34:12.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.744+0000 7fa39effd640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa3880a3dd0 0x7fa3880a6290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa394000c00 tx=0x7fa394009610 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:12.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.746+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa39009d710 con 0x7fa3a006a840 2024-04-23T18:34:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.932+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3a010a4c0 con 0x7fa3a006a840 2024-04-23T18:34:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.933+0000 7fa39cff9640 1 -- 172.21.15.37:0/849365487 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7fa39009d010 con 0x7fa3a006a840 2024-04-23T18:34:12.925 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa3880a3dd0 msgr2=0x7fa3880a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa3880a3dd0 0x7fa3880a6290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa394000c00 tx=0x7fa394009610 comp rx=0 tx=0).stop 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 msgr2=0x7fa3a0100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 0x7fa3a0100e80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa39000e9f0 tx=0x7fa39000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 shutdown_connections 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa3880a3dd0 0x7fa3880a6290 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3a0101900 0x7fa3a01a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3a0107270 0x7fa3a01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 --2- 172.21.15.37:0/849365487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a006a840 0x7fa3a0100e80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:12.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 >> 172.21.15.37:0/849365487 conn(0x7fa3a0077eb0 msgr2=0x7fa3a0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:12.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 shutdown_connections 2024-04-23T18:34:12.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:12.935+0000 7fa3a6035640 1 -- 172.21.15.37:0/849365487 wait complete. 2024-04-23T18:34:12.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-23T18:34:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 -- 172.21.15.37:0/901205405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f4103b00 msgr2=0x7f62f410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:13.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 --2- 172.21.15.37:0/901205405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f4103b00 0x7f62f410bfe0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009900 tx=0x7f62e802a510 comp rx=0 tx=0).stop 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 -- 172.21.15.37:0/901205405 shutdown_connections 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 --2- 172.21.15.37:0/901205405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f410c520 0x7f62f410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 --2- 172.21.15.37:0/901205405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f4103b00 0x7f62f410bfe0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 --2- 172.21.15.37:0/901205405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f41031e0 0x7f62f41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 -- 172.21.15.37:0/901205405 >> 172.21.15.37:0/901205405 conn(0x7f62f4077f50 msgr2=0x7f62f4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 -- 172.21.15.37:0/901205405 shutdown_connections 2024-04-23T18:34:13.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.060+0000 7f62fa98e640 1 -- 172.21.15.37:0/901205405 wait complete. 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 Processor -- start 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 -- start start 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 0x7f62f419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f4103b00 0x7f62f419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f410c520 0x7f62f41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4112210 con 0x7f62f41031e0 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4112090 con 0x7f62f410c520 2024-04-23T18:34:13.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62fa98e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4112390 con 0x7f62f4103b00 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.061+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 0x7f62f419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 0x7f62f419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48084/0 (socket says 172.21.15.37:48084) 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 -- 172.21.15.37:0/257039094 learned_addr learned my addr 172.21.15.37:0/257039094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f8f2b640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f410c520 0x7f62f41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f37fe640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f4103b00 0x7f62f419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 -- 172.21.15.37:0/257039094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f4103b00 msgr2=0x7f62f419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f4103b00 0x7f62f419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:13.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 -- 172.21.15.37:0/257039094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f410c520 msgr2=0x7f62f41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f410c520 0x7f62f41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 -- 172.21.15.37:0/257039094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f4198d30 con 0x7f62f41031e0 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f3fff640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 0x7f62f419e400 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f62e400d950 tx=0x7f62e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4017070 con 0x7f62f41031e0 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f4199020 con 0x7f62f41031e0 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e4009c30 con 0x7f62f41031e0 2024-04-23T18:34:13.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e400f620 con 0x7f62f41031e0 2024-04-23T18:34:13.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.062+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f41a52c0 con 0x7f62f41031e0 2024-04-23T18:34:13.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.063+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f4068d70 con 0x7f62f41031e0 2024-04-23T18:34:13.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.064+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62e4010460 con 0x7f62f41031e0 2024-04-23T18:34:13.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.065+0000 7f62f17fa640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62dc0a3d20 0x7f62dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:13.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.065+0000 7f62f37fe640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62dc0a3d20 0x7f62dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:13.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.065+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f62e40d0570 con 0x7f62f41031e0 2024-04-23T18:34:13.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.066+0000 7f62f37fe640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62dc0a3d20 0x7f62dc0a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62e8031440 tx=0x7f62e80095f0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:13.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.069+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62e409cb70 con 0x7f62f41031e0 2024-04-23T18:34:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.306+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f62f41076a0 con 0x7f62f41031e0 2024-04-23T18:34:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.315+0000 7f62f17fa640 1 -- 172.21.15.37:0/257039094 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746059 (secure 0 0 0) 0x7f62e4009de0 con 0x7f62f41031e0 2024-04-23T18:34:13.322 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:13.323 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:34:13.323 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-23T18:34:13.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.323 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 15270, 2024-04-23T18:34:13.323 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-04-23T18:34:13.323 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-04-23T18:34:13.324 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:13.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:13.324 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-04-23T18:34:13.324 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 816900776 2024-04-23T18:34:13.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-23T18:34:13.325 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-04-23T18:34:13.325 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 816900776 2024-04-23T18:34:13.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.325 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:13.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.326 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/816900776", 2024-04-23T18:34:13.326 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-04-23T18:20:30.552883+0000", 2024-04-23T18:34:13.326 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-04-23T18:34:13.326 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-04-23T18:34:13.326 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-04-23T18:34:13.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.327 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 15264, 2024-04-23T18:34:13.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-04-23T18:34:13.327 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:34:13.327 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-23T18:34:13.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-23T18:34:13.328 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.328 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.328 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.328 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:13.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:13.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:13.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:34:13.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.331 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.332 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.332 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.332 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.333 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.333 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.333 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.333 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.335 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.336 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.337 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-23T18:34:13.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.343 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-23T18:34:13.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-23T18:34:13.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-23T18:34:13.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-23T18:34:13.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:34:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-23T18:34:13.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:34:13.361 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.363 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:13.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.368 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.372 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.377 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-23T18:34:13.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-23T18:34:13.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-23T18:34:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:34:13.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:34:13.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-23T18:34:13.402 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-23T18:34:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:34:13.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-23T18:34:13.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-23T18:34:13.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:34:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-23T18:34:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-23T18:34:13.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:13.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-23T18:34:13.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-23T18:34:13.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-23T18:34:13.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:34:13.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.438 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:34:13.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.443 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:34:13.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.446 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.449 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-23T18:34:13.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:34:13.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:34:13.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-23T18:34:13.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-23T18:34:13.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.465 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.467 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-23T18:34:13.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:34:13.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-23T18:34:13.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:34:13.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-23T18:34:13.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:34:13.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.488 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.492 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.496 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-23T18:34:13.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:34:13.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:34:13.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:34:13.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:34:13.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:34:13.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.521 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:34:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-23T18:34:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:34:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:34:13.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:34:13.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:34:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.552 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:34:13.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.562 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:34:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:34:13.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.573 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.578 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-23T18:34:13.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.588 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.592 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:13.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.603 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-23T18:34:13.606 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.610 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-23T18:34:13.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.614 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.615 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-23T18:34:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:13.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.640 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.641 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-23T18:34:13.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-23T18:34:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-23T18:34:13.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:34:13.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.657 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.661 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-23T18:34:13.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-23T18:34:13.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-23T18:34:13.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:34:13.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.676 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.680 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.683 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.685 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.689 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.692 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-23T18:34:13.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.705 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.708 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.712 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-23T18:34:13.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-23T18:34:13.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:34:13.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-23T18:34:13.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:34:13.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.726 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.735 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.736 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.741 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-23T18:34:13.744 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.746 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.747 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.754 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.757 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.761 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-23T18:34:13.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.765 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.768 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.772 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.776 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.785 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.790 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:13.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-23T18:34:13.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.804 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.805 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-23T18:34:13.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:13.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.821 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:34:13.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-23T18:34:13.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:13.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-23T18:34:13.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-23T18:34:13.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.840 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.844 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.850 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.857 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.861 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:13.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.868 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.874 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.877 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.879 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.883 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.888 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.894 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-23T18:34:13.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-23T18:34:13.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.904 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-23T18:34:13.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-23T18:34:13.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.915 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:34:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.923 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.925 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.927 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.936 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.941 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.945 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.953 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.958 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.961 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-23T18:34:13.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:34:13.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-23T18:34:13.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-23T18:34:13.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-23T18:34:13.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.984 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.987 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:13.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-23T18:34:13.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:13.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:13.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:13.998 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:13.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.002 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.006 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.007 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.028 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.032 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.034 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-23T18:34:14.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-23T18:34:14.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24923, 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.044 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.048 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-23T18:34:14.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.054 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.058 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.059 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-23T18:34:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:34:14.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-23T18:34:14.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-23T18:34:14.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:34:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-23T18:34:14.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.075 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:34:14.076 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.080 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.084 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-23T18:34:14.090 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-23T18:34:14.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.098 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-23T18:34:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:34:14.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-23T18:34:14.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-23T18:34:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.109 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-23T18:34:14.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-23T18:34:14.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:34:14.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.119 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-23T18:34:14.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-23T18:34:14.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:34:14.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-23T18:34:14.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.128 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:34:14.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.132 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.135 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:34:14.136 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.137 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:34:14.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.142 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-23T18:34:14.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-23T18:34:14.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:34:14.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-23T18:34:14.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.159 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.162 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.167 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-23T18:34:14.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-23T18:34:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:34:14.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:34:14.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-23T18:34:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.190 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-23T18:34:14.198 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:34:14.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.210 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-23T18:34:14.216 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:34:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:34:14.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.235 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:14.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:34:14.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:34:14.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:34:14.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.261 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:34:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.267 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:34:14.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:34:14.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:34:14.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-23T18:34:14.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:34:14.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.299 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.301 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.305 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:34:14.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.313 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-23T18:34:14.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.318 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.319 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.320 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.324 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.327 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.333 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-23T18:34:14.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-23T18:34:14.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-23T18:34:14.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:34:14.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-23T18:34:14.348 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.350 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.354 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.357 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.363 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-23T18:34:14.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.371 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.374 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-23T18:34:14.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-23T18:34:14.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.383 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-23T18:34:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-23T18:34:14.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.390 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.394 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.397 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.399 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.402 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.403 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-23T18:34:14.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.412 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-23T18:34:14.419 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.421 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.424 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.427 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-23T18:34:14.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:34:14.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-23T18:34:14.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:34:14.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.440 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.449 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.458 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.462 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-23T18:34:14.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.469 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.470 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.473 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.474 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-23T18:34:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-23T18:34:14.477 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.480 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.485 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.488 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.490 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.494 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.499 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.503 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:14.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.509 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-23T18:34:14.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:34:14.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.513 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.519 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.529 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.531 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.535 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-23T18:34:14.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-23T18:34:14.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-23T18:34:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-23T18:34:14.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.550 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-23T18:34:14.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-23T18:34:14.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-23T18:34:14.552 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.554 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-23T18:34:14.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-23T18:34:14.567 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.569 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.572 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.575 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.577 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.585 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.589 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.594 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.601 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.605 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.609 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-23T18:34:14.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-23T18:34:14.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.622 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-23T18:34:14.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-23T18:34:14.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-23T18:34:14.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.638 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.652 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.657 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.661 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-23T18:34:14.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.670 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.674 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-23T18:34:14.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:34:14.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-23T18:34:14.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.693 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-23T18:34:14.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.701 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.702 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-23T18:34:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.724 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.728 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-23T18:34:14.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-23T18:34:14.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.736 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.745 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.748 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-23T18:34:14.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-04-23T18:34:14.758 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-23T18:34:14.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.763 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.768 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-23T18:34:14.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-23T18:34:14.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.773 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-23T18:34:14.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-23T18:34:14.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-23T18:34:14.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:34:14.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-23T18:34:14.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.790 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-23T18:34:14.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:34:14.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.798 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.802 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-23T18:34:14.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-23T18:34:14.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-23T18:34:14.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-23T18:34:14.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.821 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-23T18:34:14.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.826 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.827 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.828 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.829 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-23T18:34:14.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:34:14.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-23T18:34:14.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.840 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:34:14.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:34:14.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-23T18:34:14.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:34:14.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.858 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.868 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:34:14.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.873 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.878 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.885 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:34:14.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-23T18:34:14.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-23T18:34:14.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.900 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-23T18:34:14.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:34:14.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-23T18:34:14.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-23T18:34:14.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:34:14.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:34:14.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:34:14.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.922 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:14.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:14.928 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:14.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-23T18:34:14.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:14.935 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.940 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:34:14.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.948 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:34:14.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:34:14.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:34:14.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:14.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.966 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:34:14.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.975 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.976 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:34:14.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.983 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.988 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.993 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:34:14.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:14.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:34:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.006 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:34:15.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-23T18:34:15.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.019 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.022 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.026 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-23T18:34:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-23T18:34:15.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:15.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.042 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.048 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-23T18:34:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:34:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-23T18:34:15.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:34:15.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.061 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-23T18:34:15.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.068 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.069 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.071 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.072 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-23T18:34:15.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-23T18:34:15.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.084 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.089 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.096 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-23T18:34:15.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:15.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.103 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.107 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.113 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-23T18:34:15.116 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.118 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.122 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.123 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-23T18:34:15.125 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.130 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.133 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-23T18:34:15.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:34:15.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.141 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.145 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-23T18:34:15.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:34:15.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.153 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-23T18:34:15.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:34:15.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.157 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.159 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.163 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.168 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.172 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.177 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.182 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.184 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-23T18:34:15.185 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.187 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.188 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.194 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.212 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.222 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:15.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:15.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.229 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.238 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.244 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-23T18:34:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:34:15.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-23T18:34:15.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-23T18:34:15.270 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-23T18:34:15.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-23T18:34:15.273 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.275 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.276 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.279 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:34:15.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.285 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.288 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-23T18:34:15.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.293 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.296 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.306 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.311 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.314 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.316 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.320 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.322 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-23T18:34:15.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.327 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.331 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-23T18:34:15.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-23T18:34:15.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.361 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.369 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.371 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.375 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.380 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.383 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.384 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.388 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:15.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.397 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-23T18:34:15.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:34:15.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.406 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.407 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-23T18:34:15.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-23T18:34:15.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-23T18:34:15.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-23T18:34:15.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.423 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.429 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-23T18:34:15.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.439 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.443 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.446 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.458 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T18:34:15.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.466 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-23T18:34:15.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.469 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-23T18:34:15.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.478 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-23T18:34:15.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-23T18:34:15.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-04-23T18:34:15.482 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-23T18:34:15.483 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-04-23T18:34:15.484 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-23T18:34:15.485 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-23T18:34:15.486 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-04-23T18:34:15.487 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-23T18:34:15.488 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-23T18:34:15.489 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.490 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 91, 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-23T18:34:15.491 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1142444412 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2424400950 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-23T18:34:15.493 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1744476031 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 491882083 2024-04-23T18:34:15.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:15.496 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3161310203 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-23T18:34:15.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1016735457 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-23T18:34:15.498 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:34:15.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62dc0a3d20 msgr2=0x7f62dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62dc0a3d20 0x7f62dc0a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f62e8031440 tx=0x7f62e80095f0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 msgr2=0x7f62f419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 0x7f62f419e400 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f62e400d950 tx=0x7f62e400de10 comp rx=0 tx=0).stop 2024-04-23T18:34:15.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 shutdown_connections 2024-04-23T18:34:15.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f62dc0a3d20 0x7f62dc0a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62f410c520 0x7f62f41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62f4103b00 0x7f62f419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 --2- 172.21.15.37:0/257039094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62f41031e0 0x7f62f419e400 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 >> 172.21.15.37:0/257039094 conn(0x7f62f4077f50 msgr2=0x7f62f40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 shutdown_connections 2024-04-23T18:34:15.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:13.320+0000 7f62fa98e640 1 -- 172.21.15.37:0/257039094 wait complete. 2024-04-23T18:34:15.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:13 smithi037 ceph-mon[29404]: from='client.16989 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:15.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:13 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1672239123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:13 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2102474959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:13 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/849365487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.502 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:13 smithi173 ceph-mon[32906]: from='client.16989 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:15.502 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:13 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1672239123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:13 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2102474959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:13 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/849365487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:14 smithi173 ceph-mon[32906]: pgmap v433: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:14 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/257039094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:13 smithi172 ceph-mon[32456]: from='client.16989 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:13 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1672239123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:13 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2102474959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.503 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:13 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/849365487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:34:15.504 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:14 smithi172 ceph-mon[32456]: pgmap v433: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-04-23T18:34:15.504 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:14 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/257039094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:34:15.510 DEBUG:tasks.check_counter:Getting stats from a 2024-04-23T18:34:15.511 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-23T18:34:15.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:14 smithi037 ceph-mon[29404]: pgmap v433: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-04-23T18:34:15.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:14 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/257039094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T18:34:15.769 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:16.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:16 smithi037 ceph-mon[29404]: pgmap v434: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 299 op/s 2024-04-23T18:34:16.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:16 smithi172 ceph-mon[32456]: pgmap v434: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 299 op/s 2024-04-23T18:34:16.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:16 smithi173 ceph-mon[32906]: pgmap v434: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 299 op/s 2024-04-23T18:34:17.521 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:34:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-23T18:34:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001463776, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T18:34:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000319410, 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 265, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 254, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1602080, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 156560, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.048324921, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.016273885, 2024-04-23T18:34:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.035262441, 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005086888, 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 253, 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012137269, 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000047973 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1602080, 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 156560 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-23T18:34:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 197, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3312, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 190848, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.016079593, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.011178016, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002298286, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 195, 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.015366522, 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000078802 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-23T18:34:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 190848 2024-04-23T18:34:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-23T18:34:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-23T18:34:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 150008, 2024-04-23T18:34:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-23T18:34:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 267, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1564694, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 267, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 267, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1564694, 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-23T18:34:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:17.540 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:34:18.273 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-23T18:34:18.273 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-23T18:34:18.273 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-23T18:34:18.273 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-23T18:34:18.274 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-23T18:34:18.274 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-23T18:34:18.274 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-23T18:34:18.274 DEBUG:tasks.check_counter:Getting stats from d 2024-04-23T18:34:18.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-23T18:34:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:18.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:18 smithi172 ceph-mon[32456]: pgmap v435: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:18.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:18 smithi173 ceph-mon[32906]: pgmap v435: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:18.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:18 smithi037 ceph-mon[29404]: pgmap v435: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:19.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:19 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:19.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:19 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:19.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:19 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:20.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:20 smithi037 ceph-mon[29404]: pgmap v436: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:20.433 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:34:20.433 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:20.433 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-23T18:34:20.433 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-23T18:34:20.433 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2992, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001989020, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000042400, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000617424, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:20.434 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000716875, 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000238958 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:20.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 271, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 260, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1603712, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 160400, 2024-04-23T18:34:20.436 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.050647854, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.015934779, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.038872378, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005094671, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 259, 2024-04-23T18:34:20.437 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011327121, 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000043734 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.438 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1603712, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 160400 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 199, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 191104, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:20.439 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015869711, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.011542016, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001806111, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 196, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011965320, 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000061047 2024-04-23T18:34:20.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 191104 2024-04-23T18:34:20.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:20.442 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:20.443 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:20.444 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148995, 2024-04-23T18:34:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:20.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-23T18:34:20.448 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-23T18:34:20.449 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 273, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1565474, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 273, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 273, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1565474, 2024-04-23T18:34:20.450 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-23T18:34:20.451 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:20.451 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:20.451 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:20.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:20.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:34:20.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:20 smithi172 ceph-mon[32456]: pgmap v436: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:20.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:20 smithi173 ceph-mon[32906]: pgmap v436: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:21.122 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-23T18:34:21.122 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-23T18:34:21.122 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-23T18:34:21.122 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-23T18:34:21.123 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-23T18:34:21.123 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-23T18:34:21.123 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-23T18:34:21.123 DEBUG:tasks.check_counter:Getting stats from g 2024-04-23T18:34:21.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-23T18:34:21.376 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:22.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:22 smithi037 ceph-mon[29404]: pgmap v437: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:22.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:22 smithi172 ceph-mon[32456]: pgmap v437: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:22.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:22 smithi173 ceph-mon[32906]: pgmap v437: 97 pgs: 97 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 78 op/s 2024-04-23T18:34:23.100 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:34:23.100 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:23.100 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 268, 2024-04-23T18:34:23.100 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 257, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1602896, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 158480, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.049618045, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.014967739, 2024-04-23T18:34:23.101 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.038293206, 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005082146, 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 256, 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011486592, 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000044869 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:23.102 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1602896, 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 158480 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 196, 2024-04-23T18:34:23.103 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 190576, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.016510065, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.011726710, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001975303, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:23.104 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 193, 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012017459, 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000062266 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:23.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 190576 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2992, 2024-04-23T18:34:23.106 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000714801, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000039334, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000643002, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-23T18:34:23.107 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000923975, 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000307991 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.108 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.109 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-23T18:34:23.110 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:23.110 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:23.110 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:23.110 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:23.110 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:23.111 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:23.111 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:23.111 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:23.111 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:23.112 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:23.113 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-23T18:34:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148994, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:23.115 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:23.116 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-23T18:34:23.116 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-23T18:34:23.116 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:23.116 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:23.116 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-23T18:34:23.117 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:23.118 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 270, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1565084, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 270, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-23T18:34:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 270, 2024-04-23T18:34:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1565084, 2024-04-23T18:34:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-23T18:34:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:23.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:23.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:23.121 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:34:23.795 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-23T18:34:23.795 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-23T18:34:23.795 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-23T18:34:23.795 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-23T18:34:23.795 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-23T18:34:23.796 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-23T18:34:23.796 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-23T18:34:23.796 DEBUG:tasks.check_counter:Getting stats from j 2024-04-23T18:34:23.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-23T18:34:24.050 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:34:24.405 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:24 smithi173 ceph-mon[32906]: pgmap v438: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-04-23T18:34:24.406 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:24 smithi173 ceph-mon[32906]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:24.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:24 smithi037 ceph-mon[29404]: pgmap v438: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-04-23T18:34:24.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:24 smithi037 ceph-mon[29404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:24.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:24 smithi172 ceph-mon[32456]: pgmap v438: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 84 op/s 2024-04-23T18:34:24.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:24 smithi172 ceph-mon[32456]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-23T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-23T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001244208, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000249645, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 273, 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 262, 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1604256, 2024-04-23T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 161680, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.052650425, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.016352747, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.039687293, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005220834, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 261, 2024-04-23T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011726326, 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000044928 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1604256, 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 161680 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 202, 2024-04-23T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3312, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 191728, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015875297, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.010743696, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002481696, 2024-04-23T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 200, 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012426020, 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000062130 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 191728 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-23T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-23T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148977, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-23T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-23T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-23T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-23T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 275, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1565734, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 275, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 275, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1565734, 2024-04-23T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-23T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-23T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-23T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-23T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-23T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-23T18:34:26.290 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:26 smithi037 ceph-mon[29404]: pgmap v439: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 608 KiB/s wr, 39 op/s 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-23T18:34:26.319 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-23T18:34:26.320 DEBUG:tasks.check_counter:Getting stats from b 2024-04-23T18:34:26.320 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-23T18:34:26.571 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:34:26.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:26 smithi172 ceph-mon[32456]: pgmap v439: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 608 KiB/s wr, 39 op/s 2024-04-23T18:34:26.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:26 smithi173 ceph-mon[32906]: pgmap v439: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 608 KiB/s wr, 39 op/s 2024-04-23T18:34:28.268 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-23T18:34:28.269 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:28.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 133571, 2024-04-23T18:34:28.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 88767, 2024-04-23T18:34:28.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 41870034, 2024-04-23T18:34:28.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 45338018, 2024-04-23T18:34:28.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 148, 2024-04-23T18:34:28.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 8, 2024-04-23T18:34:28.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 6.521611425, 2024-04-23T18:34:28.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 2.858855017, 2024-04-23T18:34:28.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 4.680893500, 2024-04-23T18:34:28.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.185869096, 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 88765, 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.104875526, 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000034978 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 208411, 2024-04-23T18:34:28.271 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.025627839, 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000122 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 43952, 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 4176 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 73297, 2024-04-23T18:34:28.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 95477, 2024-04-23T18:34:28.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 22641453, 2024-04-23T18:34:28.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 98079520, 2024-04-23T18:34:28.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 17, 2024-04-23T18:34:28.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-04-23T18:34:28.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 3.825241927, 2024-04-23T18:34:28.274 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 1.572884245, 2024-04-23T18:34:28.274 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 2.297044723, 2024-04-23T18:34:28.274 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.328979411, 2024-04-23T18:34:28.274 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:28.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 95474, 2024-04-23T18:34:28.274 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 4.142958324, 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000043393 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 144553, 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.019952078, 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000138 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 101952, 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 551584 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 9149, 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 8706, 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 4632687, 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 21610798, 2024-04-23T18:34:28.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 18, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.725599971, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.264410121, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.259866549, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.199356686, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 8706, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 1.301572681, 2024-04-23T18:34:28.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000149502 2024-04-23T18:34:28.278 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.278 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:28.278 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 6267, 2024-04-23T18:34:28.278 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001068642, 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000170 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2208064, 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 310464 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.279 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 16158, 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 9.492252727, 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000587464 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-04-23T18:34:28.280 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 6928, 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.373243831, 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000053874 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-04-23T18:34:28.281 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 44405, 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 36116, 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 36116, 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 106.266134864, 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002942356 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-04-23T18:34:28.282 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 8289, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 20, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 588, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 1, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 740, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 79, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 523, 2024-04-23T18:34:28.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 138, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 625, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 574, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 241, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 241, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 12, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 74244, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 35700, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 8279, 2024-04-23T18:34:28.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 192, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 474, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 58509, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 13, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 343, 2024-04-23T18:34:28.285 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 9, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 639, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 662, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 3547397, 2024-04-23T18:34:28.286 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:34:28.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 6519, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 23585, 2024-04-23T18:34:28.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 15980, 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 4499, 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 3127, 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 34484 2024-04-23T18:34:28.289 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 633, 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 945, 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 525, 2024-04-23T18:34:28.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 957, 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 755, 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 24, 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 3504, 2024-04-23T18:34:28.291 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 3480, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-04-23T18:34:28.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 7, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 228, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 15, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 1, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-04-23T18:34:28.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-04-23T18:34:28.294 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.294 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-04-23T18:34:28.294 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-04-23T18:34:28.294 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 38661, 2024-04-23T18:34:28.294 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 38463, 2024-04-23T18:34:28.294 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 38463, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 198, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 807, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 78, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 76, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 76, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 1, 2024-04-23T18:34:28.295 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 1, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 50170079, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 50753470, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 7735, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 17.094948194, 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002210077 2024-04-23T18:34:28.296 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 403, 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 4357, 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 3954, 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 140, 2024-04-23T18:34:28.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 505, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 365, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 751, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 8063, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 7312, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 241, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 4994, 2024-04-23T18:34:28.298 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 4753, 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 68780, 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 231800 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 65639, 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 1395, 2024-04-23T18:34:28.299 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 44405, 2024-04-23T18:34:28.300 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 608, 2024-04-23T18:34:28.300 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 8495, 2024-04-23T18:34:28.300 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-04-23T18:34:28.300 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3590, 2024-04-23T18:34:28.300 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 8.233270013, 2024-04-23T18:34:28.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002293389 2024-04-23T18:34:28.301 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.301 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-04-23T18:34:28.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 146, 2024-04-23T18:34:28.301 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.035711131, 2024-04-23T18:34:28.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000244596 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-04-23T18:34:28.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 5898, 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 21.667241871, 2024-04-23T18:34:28.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.003673659 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-04-23T18:34:28.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.305 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-04-23T18:34:28.306 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.307 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.307 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 14, 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.002864277, 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000204591 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.308 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 7, 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.009853403, 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001407629 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-04-23T18:34:28.309 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 14, 2024-04-23T18:34:28.310 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 49.213829722, 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 3.515273551 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000230221, 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000230221 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.311 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-04-23T18:34:28.312 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 22450, 2024-04-23T18:34:28.312 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 10.554023299, 2024-04-23T18:34:28.312 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000470112 2024-04-23T18:34:28.312 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.312 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-04-23T18:34:28.312 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 474, 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 1.013371534, 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002137914 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.314 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 8, 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.971499533, 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.496437441 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-04-23T18:34:28.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3, 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.006877600, 2024-04-23T18:34:28.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002292533 2024-04-23T18:34:28.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.317 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-04-23T18:34:28.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.317 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.318 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-04-23T18:34:28.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.318 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 7, 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.018977024, 2024-04-23T18:34:28.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002711003 2024-04-23T18:34:28.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.320 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-04-23T18:34:28.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.320 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.321 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-04-23T18:34:28.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3504, 2024-04-23T18:34:28.321 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 11.538385236, 2024-04-23T18:34:28.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.003292918 2024-04-23T18:34:28.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.322 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:34:28.322 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:34:28.322 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-04-23T18:34:28.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.323 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.323 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-04-23T18:34:28.323 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-04-23T18:34:28.323 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 5, 2024-04-23T18:34:28.324 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 3, 2024-04-23T18:34:28.324 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-04-23T18:34:28.324 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-04-23T18:34:28.324 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 4807, 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 2403, 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 819, 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:28.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:28.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:28.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:28.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:28.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 793811, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 698, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 88, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 1, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-23T18:34:28.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8232, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 172, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:28.331 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 2079417, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 28341, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-23T18:34:28.332 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 17620, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 52008285, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 37, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 17620, 2024-04-23T18:34:28.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 17620, 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 102.659387141, 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.005826298 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 17620, 2024-04-23T18:34:28.334 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 26468 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 17620, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 441, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 17179, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 557, 2024-04-23T18:34:28.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 3365, 2024-04-23T18:34:28.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 421, 2024-04-23T18:34:28.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 8156, 2024-04-23T18:34:28.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 469, 2024-04-23T18:34:28.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-04-23T18:34:28.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-04-23T18:34:28.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 1, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 4013, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 6730, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:34:28.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:34:28.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-04-23T18:34:28.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 110, 2024-04-23T18:34:28.340 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 23, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 12, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 2080, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 40, 2024-04-23T18:34:28.341 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 635 2024-04-23T18:34:28.342 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.342 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-04-23T18:34:28.342 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-04-23T18:34:28.342 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:34:28.342 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 284 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 10, 2024-04-23T18:34:28.343 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 10, 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 3480, 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 3480, 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:28.344 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 216013, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 49620599, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 216013, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 216013, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 49620599, 2024-04-23T18:34:28.345 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-04-23T18:34:28.346 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 17620, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 52361317, 2024-04-23T18:34:28.347 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 17620, 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 52361317, 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.348 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:28.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 17620, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 17620, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 17620, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 17620, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.350 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:28.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 3480, 2024-04-23T18:34:28.352 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 353352, 2024-04-23T18:34:28.352 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:28.352 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 3480, 2024-04-23T18:34:28.352 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:28.352 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:28.352 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 421, 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 353352, 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x561d9ba4e500": { 2024-04-23T18:34:28.353 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 38661, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 46559166, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 38661, 2024-04-23T18:34:28.354 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 7735, 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 46559166, 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.355 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:28.356 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-23T18:34:28.356 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:28 smithi172 ceph-mon[32456]: pgmap v440: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-23T18:34:28.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:28 smithi173 ceph-mon[32906]: pgmap v440: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-23T18:34:28.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:28 smithi037 ceph-mon[29404]: pgmap v440: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 1.1 MiB/s wr, 60 op/s 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds.exported=13 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=633 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=945 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=14 2024-04-23T18:34:28.915 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=8 2024-04-23T18:34:28.915 DEBUG:tasks.check_counter:Getting stats from e 2024-04-23T18:34:28.916 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-23T18:34:29.162 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:34:30.326 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:30 smithi172 ceph-mon[32456]: pgmap v441: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 778 KiB/s wr, 38 op/s 2024-04-23T18:34:30.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:30 smithi173 ceph-mon[32906]: pgmap v441: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 778 KiB/s wr, 38 op/s 2024-04-23T18:34:30.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:30 smithi037 ceph-mon[29404]: pgmap v441: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 778 KiB/s wr, 38 op/s 2024-04-23T18:34:30.767 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-23T18:34:30.768 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:30.768 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1135, 2024-04-23T18:34:30.768 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1135, 2024-04-23T18:34:30.769 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 168067313, 2024-04-23T18:34:30.769 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 360170, 2024-04-23T18:34:30.769 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 8, 2024-04-23T18:34:30.770 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:30.770 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.165009155, 2024-04-23T18:34:30.770 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.026651442, 2024-04-23T18:34:30.771 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.105347000, 2024-04-23T18:34:30.771 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.024698754, 2024-04-23T18:34:30.771 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:30.771 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1134, 2024-04-23T18:34:30.772 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.033611880, 2024-04-23T18:34:30.772 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000029640 2024-04-23T18:34:30.772 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.772 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:30.773 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.773 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.773 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.773 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.774 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-23T18:34:30.774 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T18:34:30.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.774 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:30.775 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 811, 2024-04-23T18:34:30.775 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1009, 2024-04-23T18:34:30.775 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 117772254, 2024-04-23T18:34:30.775 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 772531, 2024-04-23T18:34:30.775 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 9, 2024-04-23T18:34:30.776 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-04-23T18:34:30.776 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.146801606, 2024-04-23T18:34:30.776 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.024991662, 2024-04-23T18:34:30.776 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.091821376, 2024-04-23T18:34:30.777 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.021760511, 2024-04-23T18:34:30.777 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:30.777 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1006, 2024-04-23T18:34:30.777 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.043345970, 2024-04-23T18:34:30.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000043087 2024-04-23T18:34:30.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.778 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:30.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.779 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.779 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:30.780 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 515264 2024-04-23T18:34:30.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.780 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:30.780 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2109, 2024-04-23T18:34:30.781 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2070, 2024-04-23T18:34:30.781 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 140695986, 2024-04-23T18:34:30.781 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 732742, 2024-04-23T18:34:30.781 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:30.782 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:30.782 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.287030493, 2024-04-23T18:34:30.782 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.061164857, 2024-04-23T18:34:30.782 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.160344717, 2024-04-23T18:34:30.783 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.061520838, 2024-04-23T18:34:30.783 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:30.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2069, 2024-04-23T18:34:30.783 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.073166104, 2024-04-23T18:34:30.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000035363 2024-04-23T18:34:30.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.784 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:30.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.785 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.785 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.785 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1752352, 2024-04-23T18:34:30.785 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 166384 2024-04-23T18:34:30.786 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.786 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-23T18:34:30.786 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-04-23T18:34:30.787 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-23T18:34:30.787 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.787 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-04-23T18:34:30.787 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-23T18:34:30.788 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-23T18:34:30.788 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 6055, 2024-04-23T18:34:30.788 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.149354124, 2024-04-23T18:34:30.788 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000024666 2024-04-23T18:34:30.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.789 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.789 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-04-23T18:34:30.789 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-23T18:34:30.790 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-23T18:34:30.790 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.790 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.790 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.791 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.791 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.791 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-04-23T18:34:30.791 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-04-23T18:34:30.792 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-04-23T18:34:30.792 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-04-23T18:34:30.792 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.792 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.793 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.793 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.793 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-04-23T18:34:30.793 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-04-23T18:34:30.794 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-04-23T18:34:30.794 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-04-23T18:34:30.794 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-04-23T18:34:30.794 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-04-23T18:34:30.795 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-04-23T18:34:30.795 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 11420, 2024-04-23T18:34:30.795 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 184, 2024-04-23T18:34:30.795 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 11236, 2024-04-23T18:34:30.796 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-04-23T18:34:30.796 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 11156, 2024-04-23T18:34:30.796 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-04-23T18:34:30.796 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-04-23T18:34:30.797 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-04-23T18:34:30.797 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 12, 2024-04-23T18:34:30.797 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-04-23T18:34:30.797 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-04-23T18:34:30.798 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-04-23T18:34:30.798 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-04-23T18:34:30.798 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:34:30.798 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-04-23T18:34:30.799 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-04-23T18:34:30.799 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-04-23T18:34:30.799 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-04-23T18:34:30.799 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-04-23T18:34:30.800 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-04-23T18:34:30.800 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-04-23T18:34:30.800 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-04-23T18:34:30.800 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-04-23T18:34:30.801 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:34:30.801 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-04-23T18:34:30.801 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 0, 2024-04-23T18:34:30.801 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 0, 2024-04-23T18:34:30.802 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-04-23T18:34:30.802 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:34:30.802 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-04-23T18:34:30.802 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:34:30.803 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:34:30.803 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:34:30.803 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:34:30.803 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:34:30.804 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-04-23T18:34:30.804 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:34:30.804 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:34:30.804 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:34:30.805 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:34:30.805 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:34:30.805 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:34:30.805 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-04-23T18:34:30.806 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:34:30.806 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-04-23T18:34:30.806 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-04-23T18:34:30.806 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.807 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-04-23T18:34:30.807 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-04-23T18:34:30.807 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-04-23T18:34:30.807 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-04-23T18:34:30.808 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-04-23T18:34:30.808 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-04-23T18:34:30.808 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-04-23T18:34:30.808 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-04-23T18:34:30.809 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:34:30.809 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-04-23T18:34:30.809 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-04-23T18:34:30.809 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-04-23T18:34:30.810 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-04-23T18:34:30.810 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-04-23T18:34:30.810 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:34:30.810 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:34:30.811 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-04-23T18:34:30.811 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-04-23T18:34:30.811 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:34:30.811 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:34:30.812 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:34:30.812 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-04-23T18:34:30.812 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-04-23T18:34:30.812 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:34:30.813 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-04-23T18:34:30.813 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-04-23T18:34:30.813 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.813 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-04-23T18:34:30.814 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-04-23T18:34:30.814 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-04-23T18:34:30.814 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-04-23T18:34:30.815 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-04-23T18:34:30.815 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 67017, 2024-04-23T18:34:30.815 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-04-23T18:34:30.815 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-04-23T18:34:30.816 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-04-23T18:34:30.816 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-04-23T18:34:30.816 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-04-23T18:34:30.816 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 127, 2024-04-23T18:34:30.816 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 7, 2024-04-23T18:34:30.817 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-04-23T18:34:30.817 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-04-23T18:34:30.817 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-04-23T18:34:30.817 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 427942480, 2024-04-23T18:34:30.818 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 427933297, 2024-04-23T18:34:30.818 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-04-23T18:34:30.818 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.819 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.819 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.819 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.819 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 67017 2024-04-23T18:34:30.820 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.820 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-04-23T18:34:30.820 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1250, 2024-04-23T18:34:30.820 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 11669, 2024-04-23T18:34:30.821 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 10419, 2024-04-23T18:34:30.821 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 77, 2024-04-23T18:34:30.821 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 178, 2024-04-23T18:34:30.821 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 101, 2024-04-23T18:34:30.822 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 11420, 2024-04-23T18:34:30.822 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 12527, 2024-04-23T18:34:30.822 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 1107, 2024-04-23T18:34:30.822 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-04-23T18:34:30.823 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-04-23T18:34:30.823 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-04-23T18:34:30.823 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 72628, 2024-04-23T18:34:30.823 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215416 2024-04-23T18:34:30.824 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.824 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-04-23T18:34:30.824 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-04-23T18:34:30.824 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-04-23T18:34:30.825 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-04-23T18:34:30.825 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-04-23T18:34:30.825 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-04-23T18:34:30.825 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-04-23T18:34:30.826 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.826 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.826 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.827 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.827 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-04-23T18:34:30.827 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.827 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.828 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.828 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.828 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-04-23T18:34:30.828 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.829 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.829 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.829 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.829 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-04-23T18:34:30.830 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.830 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.830 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.830 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.831 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-04-23T18:34:30.831 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.831 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.832 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.832 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.832 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-04-23T18:34:30.832 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.833 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.833 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.833 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.833 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-04-23T18:34:30.834 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.834 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.834 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.834 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.835 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-04-23T18:34:30.835 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.835 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.835 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.835 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.836 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-04-23T18:34:30.836 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.836 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.837 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.837 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.837 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-04-23T18:34:30.837 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.837 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.838 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.838 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.838 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-04-23T18:34:30.839 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.839 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.839 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.839 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.841 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-04-23T18:34:30.841 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.841 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.841 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.842 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.842 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-04-23T18:34:30.842 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.842 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.842 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-04-23T18:34:30.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.844 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-04-23T18:34:30.846 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.847 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.848 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-04-23T18:34:30.849 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.850 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.851 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:34:30.852 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-04-23T18:34:30.853 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 4, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 3, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 2621, 2024-04-23T18:34:30.854 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:30.855 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:30.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:30.857 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-23T18:34:30.858 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 295831, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 1557, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:30.859 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8232, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 172, 2024-04-23T18:34:30.860 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 11444784, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 126998, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:30.861 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 3721, 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 21, 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 3, 2024-04-23T18:34:30.862 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 3721, 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3721, 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.059922128, 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000822338 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-04-23T18:34:30.863 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3721, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3722 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 3721, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 3721, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 2126, 2024-04-23T18:34:30.864 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 1593, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-04-23T18:34:30.865 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:34:30.866 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-04-23T18:34:30.867 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-04-23T18:34:30.868 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 110, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 22, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 11, 2024-04-23T18:34:30.869 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 11, 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-04-23T18:34:30.870 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-04-23T18:34:30.871 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.872 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 4051, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 426151316, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 4051, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:30.873 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4051, 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 426151316, 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.874 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:30.875 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 3721, 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 423750723, 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3721, 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 423750723, 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.876 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:30.877 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 3721, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 3721, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3721, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 3721, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.878 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:30.879 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:30.880 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55fc3716e6e0": { 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-23T18:34:30.881 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-04-23T18:34:30.882 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:30.883 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-23T18:34:31.581 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-23T18:34:31.582 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-23T18:34:31.582 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-23T18:34:31.582 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-23T18:34:31.582 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-23T18:34:31.582 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:34:31.582 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:34:31.582 DEBUG:tasks.check_counter:Getting stats from h 2024-04-23T18:34:31.583 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-23T18:34:31.826 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:34:32.279 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:32 smithi172 ceph-mon[32456]: pgmap v442: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 778 KiB/s wr, 38 op/s 2024-04-23T18:34:32.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:32 smithi173 ceph-mon[32906]: pgmap v442: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 778 KiB/s wr, 38 op/s 2024-04-23T18:34:32.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:32 smithi037 ceph-mon[29404]: pgmap v442: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 778 KiB/s wr, 38 op/s 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 91720, 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 136435, 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 22890139, 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 135373792, 2024-04-23T18:34:33.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 9, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 6.713513521, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 3.262358019, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 3.651266186, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.279072781, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 136438, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.464995472, 2024-04-23T18:34:33.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000018066 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 208176, 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.032087658, 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000154 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1764128, 2024-04-23T18:34:33.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 180320 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 136573, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 155986, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 31247658, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 266822643, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 15, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-04-23T18:34:33.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 7.534363099, 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 3.100985013, 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 4.512342838, 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.565603973, 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 155983, 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.678041679, 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000017168 2024-04-23T18:34:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 245260, 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.036905205, 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000150 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 127488 2024-04-23T18:34:33.453 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 153921, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 199808, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 45907087, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 250011916, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 15, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-04-23T18:34:33.454 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 7.983204633, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 3.600219072, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 5.157799873, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.390472536, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 199803, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.570101514, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000017868 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.455 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 296242, 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.045009502, 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000151 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 5088, 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 392640 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.456 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 67582, 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 44.464838320, 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000657939 2024-04-23T18:34:33.457 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 16982, 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 1.796780607, 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000105805 2024-04-23T18:34:33.458 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 99002, 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 91822, 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 91822, 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 629.261898398, 2024-04-23T18:34:33.459 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.006853062 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 7180, 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 51, 2024-04-23T18:34:33.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 4, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 8, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 1583, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 175, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 804, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 604, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 1333, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 1850, 2024-04-23T18:34:33.461 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 730, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 730, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 12, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 163586, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 110047, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 7168, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 99, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:34:33.462 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 1231, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 72783, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 15, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 1159, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 15, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 521, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-04-23T18:34:33.463 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 4, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 234, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:34:33.464 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 16, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 18833, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 33085, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:34:33.465 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 33294, 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 8589, 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 4669, 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 53300 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.466 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 258, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 6, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 4, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 102, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 1579, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 440, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 10294, 2024-04-23T18:34:33.467 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 9854, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 7, 2024-04-23T18:34:33.468 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 296, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 17, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 12, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-04-23T18:34:33.469 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 67024, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 67024, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 127, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-04-23T18:34:33.470 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 127, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 7, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 428007553, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-04-23T18:34:33.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 34156, 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 71.002361393, 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002078766 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1204, 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 11722, 2024-04-23T18:34:33.472 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 10518, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 76, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 189, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 113, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 1586, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 21724, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 20138, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 730, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 11232, 2024-04-23T18:34:33.473 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 10502, 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 82120, 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223608 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 145981, 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 29805, 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 99002, 2024-04-23T18:34:33.474 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 1904, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 60373, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 10810, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 66.586602922, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.006159722 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 20408, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 108.810160829, 2024-04-23T18:34:33.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.005331740 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.476 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 23711, 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 43.639143472, 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001840459 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.477 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-04-23T18:34:33.478 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.479 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.480 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-04-23T18:34:33.480 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.480 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.480 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-04-23T18:34:33.481 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 112, 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 20.525686180, 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.183265055 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.482 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-04-23T18:34:33.483 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.483 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.483 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.483 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.483 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-04-23T18:34:33.483 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 9, 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.029637216, 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.003293024 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 22444, 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 57.793024913, 2024-04-23T18:34:33.484 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002574987 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3207, 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 280.971353737, 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.087611897 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.485 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.486 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.002348887, 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002348887 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.487 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 50, 2024-04-23T18:34:33.488 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.090891569, 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001817831 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-04-23T18:34:33.489 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 776, 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.706667254, 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000910653 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.490 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.491 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 10294, 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 50.106381419, 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004867532 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:34:33.492 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-04-23T18:34:33.493 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 4, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 14674, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 7337, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 827, 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-04-23T18:34:33.494 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 360, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 360, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:33.495 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:33.496 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:33.497 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:33.498 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 400451, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 1207, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:33.499 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8232, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 172, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 4782098, 2024-04-23T18:34:33.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 97245, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-04-23T18:34:33.501 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 46108, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 426361769, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 33, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 46108, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 46108, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 100.389446047, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002177267 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.502 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 46108, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 46937 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 46108, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 524, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 45584, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-04-23T18:34:33.503 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-04-23T18:34:33.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-04-23T18:34:33.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 524, 2024-04-23T18:34:33.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 34680, 2024-04-23T18:34:33.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 298, 2024-04-23T18:34:33.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-04-23T18:34:33.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-04-23T18:34:33.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-04-23T18:34:33.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 4, 2024-04-23T18:34:33.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 10003, 2024-04-23T18:34:33.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-04-23T18:34:33.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-04-23T18:34:33.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-04-23T18:34:33.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-04-23T18:34:33.507 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-04-23T18:34:33.508 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 110, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 22, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 12, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-04-23T18:34:33.509 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 1076, 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 0, 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 322 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-04-23T18:34:33.510 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 487 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-04-23T18:34:33.511 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 10, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 10, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 9854, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 9854, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:33.512 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 382210, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 65623971, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 382210, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:33.513 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 382210, 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 65623971, 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.514 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:33.515 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 46108, 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 427358903, 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 46108, 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 427358903, 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.516 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-23T18:34:33.517 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 46108, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 46108, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 46108, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 46108, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.518 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 9854, 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 997134, 2024-04-23T18:34:33.519 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 9854, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 524, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 997134, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.520 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55582e5e2500": { 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 1198, 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 67024, 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 423813249, 2024-04-23T18:34:33.521 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 67024, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 34156, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 423812051, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:33.522 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:33.523 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.523 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:33.523 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-23T18:34:34.202 INFO:tasks.check_counter:Daemon mds.h mds.exported=15 2024-04-23T18:34:34.203 INFO:tasks.check_counter:Daemon mds.h mds.imported=15 2024-04-23T18:34:34.203 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=258 2024-04-23T18:34:34.203 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=6 2024-04-23T18:34:34.203 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-23T18:34:34.203 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:34:34.203 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:34:34.203 DEBUG:tasks.check_counter:Getting stats from k 2024-04-23T18:34:34.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-23T18:34:34.421 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:34 smithi037 ceph-mon[29404]: pgmap v443: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 829 KiB/s wr, 47 op/s 2024-04-23T18:34:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:34:34.565 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:34 smithi172 ceph-mon[32456]: pgmap v443: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 829 KiB/s wr, 47 op/s 2024-04-23T18:34:34.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:34 smithi173 ceph-mon[32906]: pgmap v443: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 829 KiB/s wr, 47 op/s 2024-04-23T18:34:36.172 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 0, 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2, 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 0, 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 0, 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 0, 2024-04-23T18:34:36.173 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.001217177, 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.000351548, 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:36.174 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.175 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 203, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 320, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 191808, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.012613597, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.008387469, 2024-04-23T18:34:36.176 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.001836834, 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 200, 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.021001641, 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000105008 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:36.177 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 191808 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 281, 2024-04-23T18:34:36.178 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 268, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1608432, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 164352, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 2, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.051087477, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.015849469, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.038376751, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.006109242, 2024-04-23T18:34:36.179 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 268, 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.013000982, 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000048511 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:36.180 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1608432, 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 164352 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 0, 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-23T18:34:36.181 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:36.182 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:36.183 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:36.184 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 148977, 2024-04-23T18:34:36.185 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 30, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:36.186 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 928, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 2, 2024-04-23T18:34:36.187 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-23T18:34:36.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 279, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1566357, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 279, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 279, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1566357, 2024-04-23T18:34:36.189 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-23T18:34:36.190 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-23T18:34:36.190 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-23T18:34:36.190 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-23T18:34:36.190 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:36.190 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-23T18:34:36.190 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-23T18:34:36.466 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:36 smithi172 ceph-mon[32456]: pgmap v444: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 569 KiB/s wr, 30 op/s 2024-04-23T18:34:36.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:36 smithi173 ceph-mon[32906]: pgmap v444: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 569 KiB/s wr, 30 op/s 2024-04-23T18:34:36.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:36 smithi037 ceph-mon[29404]: pgmap v444: 97 pgs: 97 active+clean; 563 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 569 KiB/s wr, 30 op/s 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-23T18:34:36.820 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-23T18:34:36.821 DEBUG:tasks.check_counter:Getting stats from c 2024-04-23T18:34:36.821 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-23T18:34:37.070 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:34:38.433 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:38 smithi173 ceph-mon[32906]: pgmap v445: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 15 MiB/s wr, 79 op/s 2024-04-23T18:34:38.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:38 smithi172 ceph-mon[32456]: pgmap v445: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 15 MiB/s wr, 79 op/s 2024-04-23T18:34:38.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:38 smithi037 ceph-mon[29404]: pgmap v445: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 15 MiB/s wr, 79 op/s 2024-04-23T18:34:39.139 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T18:34:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 1376, 2024-04-23T18:34:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1357, 2024-04-23T18:34:39.139 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 14511236, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 475752, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.139290975, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.040488981, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.049193930, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.039750552, 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:39.140 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1356, 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.050723352, 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000037406 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.141 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 290176, 2024-04-23T18:34:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 102416 2024-04-23T18:34:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 84, 2024-04-23T18:34:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 204, 2024-04-23T18:34:39.142 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 4187207, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 328436, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.015656918, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.006221102, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.006028819, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.001892129, 2024-04-23T18:34:39.143 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 203, 2024-04-23T18:34:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.009472483, 2024-04-23T18:34:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000046662 2024-04-23T18:34:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:39.144 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 303248 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 882, 2024-04-23T18:34:39.145 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 884, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 17001451, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 280483, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.072573999, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.021401317, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.025172991, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.020616992, 2024-04-23T18:34:39.146 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 882, 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.033142435, 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000037576 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.147 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-23T18:34:39.148 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3608, 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.121628645, 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000033710 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T18:34:39.149 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-23T18:34:39.150 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-04-23T18:34:39.151 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 3313, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 1834, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 1479, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 693, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-04-23T18:34:39.152 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 8, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-23T18:34:39.153 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-04-23T18:34:39.154 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 631, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 3351253, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:34:39.155 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:34:39.156 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-04-23T18:34:39.157 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-23T18:34:39.158 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:34:39.159 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:34:39.160 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-23T18:34:39.161 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 30363, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 39, 2024-04-23T18:34:39.162 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 50170079, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 51621020, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 51619019, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.163 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 30363 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 438, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 3348, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 2910, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 56, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 68, 2024-04-23T18:34:39.164 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 12, 2024-04-23T18:34:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 3313, 2024-04-23T18:34:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 3558, 2024-04-23T18:34:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 245, 2024-04-23T18:34:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-04-23T18:34:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-04-23T18:34:39.165 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 54384, 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215392 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-04-23T18:34:39.166 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-23T18:34:39.167 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-23T18:34:39.168 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.169 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.170 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.171 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.172 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-23T18:34:39.173 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-23T18:34:39.175 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.176 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.177 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.178 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.179 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.180 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.181 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.182 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.183 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-23T18:34:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 2, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 0, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 0, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 2053, 2024-04-23T18:34:39.185 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-23T18:34:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T18:34:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:39.186 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:39.187 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:39.188 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:39.189 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 223273, 2024-04-23T18:34:39.190 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 518, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:39.191 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8232, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 172, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 3471032, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 24937, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:39.192 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 2135, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 26, 2024-04-23T18:34:39.193 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 4, 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 2135, 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2135, 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.133975188, 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000531135 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-23T18:34:39.194 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2135, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2136 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 2135, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 2135, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-23T18:34:39.195 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 1186, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 946, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-23T18:34:39.196 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 2, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:34:39.197 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-23T18:34:39.198 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-23T18:34:39.199 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 110, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 16, 2024-04-23T18:34:39.200 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 7, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 7, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-23T18:34:39.201 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-23T18:34:39.202 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:39.203 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2338, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 35476234, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2338, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2338, 2024-04-23T18:34:39.204 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 35476234, 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:39.205 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2135, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 34811621, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2135, 2024-04-23T18:34:39.206 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 34811621, 2024-04-23T18:34:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.207 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:39.208 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2135, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 2135, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2135, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2135, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.209 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:39.210 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.211 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x555638b726e0": { 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:39.212 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:39.213 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:39.214 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:39.214 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:39.214 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.214 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:39.214 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T18:34:40.050 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-23T18:34:40.051 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-23T18:34:40.051 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-23T18:34:40.051 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-23T18:34:40.051 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-23T18:34:40.051 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:34:40.051 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:34:40.051 DEBUG:tasks.check_counter:Getting stats from f 2024-04-23T18:34:40.052 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-23T18:34:40.301 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:34:40.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:40 smithi173 ceph-mon[32906]: pgmap v446: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 58 op/s 2024-04-23T18:34:40.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:40 smithi172 ceph-mon[32456]: pgmap v446: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 58 op/s 2024-04-23T18:34:40.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:40 smithi037 ceph-mon[29404]: pgmap v446: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 58 op/s 2024-04-23T18:34:42.011 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T18:34:42.011 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 1824, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1826, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 34101413, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 580039, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 8, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.175419944, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.050337906, 2024-04-23T18:34:42.012 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.058073073, 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.052365974, 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1824, 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.067415103, 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000036960 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.013 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 799, 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1000, 2024-04-23T18:34:42.014 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 32433420, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 774148, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.095630198, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.025855763, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.040491845, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.021461094, 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:42.015 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 997, 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.043479360, 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000043610 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.016 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 520640 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 932, 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 891, 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 31267546, 2024-04-23T18:34:42.017 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 358276, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.120457427, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.029305439, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.074179312, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.022155547, 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:42.018 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 891, 2024-04-23T18:34:42.019 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.033111749, 2024-04-23T18:34:42.019 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000037162 2024-04-23T18:34:42.019 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.019 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:42.019 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.019 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1755888, 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 168112 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T18:34:42.020 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 5452, 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.135941264, 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000024934 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.021 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.022 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-23T18:34:42.023 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 10993, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 6, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 10987, 2024-04-23T18:34:42.024 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 10985, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 6, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-04-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-04-23T18:34:42.026 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:34:42.026 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-23T18:34:42.026 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-23T18:34:42.026 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-23T18:34:42.026 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-23T18:34:42.026 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 0, 2024-04-23T18:34:42.027 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-23T18:34:42.028 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-04-23T18:34:42.029 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-23T18:34:42.030 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-23T18:34:42.031 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-04-23T18:34:42.032 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-04-23T18:34:42.033 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 53657, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-23T18:34:42.034 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 68, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 6, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 99323749, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 99321748, 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-23T18:34:42.035 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 53657 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 477, 2024-04-23T18:34:42.036 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 10922, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 10445, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 58, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 60, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 2, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 11049, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 11227, 2024-04-23T18:34:42.037 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 178, 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 63752, 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215416 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-23T18:34:42.038 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.039 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.040 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-23T18:34:42.040 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.040 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.040 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.040 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.040 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.041 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.042 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.043 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.044 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.045 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.046 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.046 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-23T18:34:42.047 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.047 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.047 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.047 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.047 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-23T18:34:42.047 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.048 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-23T18:34:42.049 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.050 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.050 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.050 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.050 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-23T18:34:42.050 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.051 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.052 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.053 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.054 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.055 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.056 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.057 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.058 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-23T18:34:42.058 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.058 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.058 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.058 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.059 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 3, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 1, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-04-23T18:34:42.060 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-04-23T18:34:42.061 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 2052, 2024-04-23T18:34:42.061 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-23T18:34:42.061 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.061 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T18:34:42.061 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:42.061 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:42.062 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:42.063 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:42.064 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:42.065 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 257901, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 540, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:42.066 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8232, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 172, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T18:34:42.067 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 8945240, 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 55074, 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.068 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 3219, 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 21, 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 3, 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 3219, 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-23T18:34:42.069 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3219, 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.026955529, 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000629684 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3219, 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 3220 2024-04-23T18:34:42.070 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.071 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 3219, 2024-04-23T18:34:42.071 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 3219, 2024-04-23T18:34:42.071 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-04-23T18:34:42.071 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-23T18:34:42.071 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 1822, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 1395, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-23T18:34:42.072 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 1, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:34:42.073 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:34:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-23T18:34:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 110, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-23T18:34:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 22, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 11, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 1, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-23T18:34:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-23T18:34:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-04-23T18:34:42.079 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 3551, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 97463038, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 3551, 2024-04-23T18:34:42.080 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3551, 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 97463038, 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.081 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.082 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-23T18:34:42.082 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:42.082 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:42.082 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:42.082 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:42.082 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 3219, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 95148801, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3219, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 95148801, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.083 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:42.084 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 3219, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 3219, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3219, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 3219, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.085 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:42.086 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.087 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5590bb3826e0": { 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:42.088 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T18:34:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T18:34:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:42.090 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T18:34:42.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:42 smithi173 ceph-mon[32906]: pgmap v447: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 58 op/s 2024-04-23T18:34:42.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:42 smithi172 ceph-mon[32456]: pgmap v447: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 58 op/s 2024-04-23T18:34:42.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:42 smithi037 ceph-mon[29404]: pgmap v447: 97 pgs: 97 active+clean; 742 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 15 MiB/s wr, 58 op/s 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:34:42.909 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:34:42.910 DEBUG:tasks.check_counter:Getting stats from i 2024-04-23T18:34:42.910 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-23T18:34:43.149 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:34:44.457 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:44 smithi173 ceph-mon[32906]: pgmap v448: 97 pgs: 97 active+clean; 745 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 17 MiB/s wr, 108 op/s 2024-04-23T18:34:44.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:44 smithi172 ceph-mon[32456]: pgmap v448: 97 pgs: 97 active+clean; 745 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 17 MiB/s wr, 108 op/s 2024-04-23T18:34:44.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:44 smithi037 ceph-mon[29404]: pgmap v448: 97 pgs: 97 active+clean; 745 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 17 MiB/s wr, 108 op/s 2024-04-23T18:34:44.863 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T18:34:44.863 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 13403, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 13079, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 3531534, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 42113823, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 12, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.889873622, 2024-04-23T18:34:44.864 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.324151640, 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.273122511, 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.258194430, 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 13078, 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.358069855, 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000027379 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:44.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 5151, 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000921264, 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000178 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 5040, 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 8320 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.866 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 88653, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 140780, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 28370354, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 130275620, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 13, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 9, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 4.941380428, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 2.164782062, 2024-04-23T18:34:44.867 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 2.940862956, 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.245773565, 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 140777, 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.363718006, 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000016790 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:44.868 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 193443, 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.026631058, 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000137 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1767040, 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 172256 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:44.869 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 141586, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 122316, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 43557293, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 56009517, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 12, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 6.608987502, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 2.666311787, 2024-04-23T18:34:44.870 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 4.713666773, 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.268407304, 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 122315, 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.876729594, 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000015343 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 245184, 2024-04-23T18:34:44.871 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.030509247, 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000124 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 520288 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T18:34:44.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 31550, 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 16.195324205, 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000513322 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 17828, 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.102800628, 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000061857 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.874 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 60570, 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 53449, 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 53449, 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 75.090348732, 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001404897 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.875 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 7121, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 825, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 54, 2024-04-23T18:34:44.876 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 686, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 85, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 748, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 259, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 244, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 244, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 12, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 133102, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 53581, 2024-04-23T18:34:44.877 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 7121, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 188, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 1375, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 44203, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 2, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 178, 2024-04-23T18:34:44.878 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 7, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 572, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 234, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-04-23T18:34:44.879 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-23T18:34:44.880 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 492, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 35480, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 9967, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 6551, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 1664, 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 53291 2024-04-23T18:34:44.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 315, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 1713, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 929, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 1620, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 345, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 4, 2024-04-23T18:34:44.882 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 10314, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 10310, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-23T18:34:44.883 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 7, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 220, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 2, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-23T18:34:44.884 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 53662, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-23T18:34:44.885 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 53662, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 68, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 68, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 6, 2024-04-23T18:34:44.886 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 99330618, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 15874, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 29.127252067, 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001834903 2024-04-23T18:34:44.887 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 560, 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 11070, 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 10510, 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 140, 2024-04-23T18:34:44.888 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 852, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 712, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 825, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 15013, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 14188, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 239, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 10799, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 10560, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 69144, 2024-04-23T18:34:44.889 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223608 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 114241, 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 71, 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 60570, 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 1062, 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 22261, 2024-04-23T18:34:44.890 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 10314, 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 42.418417351, 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.004112702 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 211, 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.138811105, 2024-04-23T18:34:44.891 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000657872 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.892 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 12662, 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 5.876370404, 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000464094 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.893 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000195073, 2024-04-23T18:34:44.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000097536 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-23T18:34:44.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-23T18:34:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.896 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-23T18:34:44.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.898 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.899 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-23T18:34:44.900 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 18566, 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 7.973332872, 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000429458 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1373, 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.897499826, 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002110342 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.901 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.902 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-23T18:34:44.903 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.904 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 7, 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.018152096, 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002593156 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.905 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-23T18:34:44.906 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.907 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 10314, 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 15.767570005, 2024-04-23T18:34:44.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001528754 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.909 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 3, 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 1, 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-23T18:34:44.910 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 4593, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 2296, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 830, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:44.911 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:44.912 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:44.913 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:44.914 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:44.915 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 360877, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 580, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:44.916 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8232, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 172, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 2358610, 2024-04-23T18:34:44.917 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 19131, 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-23T18:34:44.918 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 28109, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 97146183, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 30, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 28109, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 28109, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 60.351676625, 2024-04-23T18:34:44.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002147058 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 28109, 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 28846 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 28109, 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 551, 2024-04-23T18:34:44.920 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 27558, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 551, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 16425, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 309, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-23T18:34:44.921 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 10344, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-04-23T18:34:44.922 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T18:34:44.923 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-23T18:34:44.924 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-23T18:34:44.925 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 110, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 22, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 12, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-23T18:34:44.926 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 861, 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 326 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-23T18:34:44.927 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 283 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 10, 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-23T18:34:44.928 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 10, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 10310, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 10310, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:44.929 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 243638, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 53509996, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 243638, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 243638, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 53509996, 2024-04-23T18:34:44.930 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:44.931 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 28109, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 98189189, 2024-04-23T18:34:44.932 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 28109, 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 98189189, 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-23T18:34:44.933 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 28109, 2024-04-23T18:34:44.934 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 28109, 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 28109, 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 28109, 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 10310, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1043006, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:44.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 10310, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 551, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1043006, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.937 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x55c4e77f6500": { 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 6869, 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 53662, 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 95136314, 2024-04-23T18:34:44.938 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 53662, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 15874, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 95129445, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:44.939 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:44.940 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.940 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:44.940 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds.exported=2 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=315 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1713 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-23T18:34:45.661 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T18:34:45.662 DEBUG:tasks.check_counter:Getting stats from l 2024-04-23T18:34:45.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid af48a038-019c-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-23T18:34:45.899 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:34:46.320 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:46 smithi173 ceph-mon[32906]: pgmap v449: 97 pgs: 97 active+clean; 745 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 17 MiB/s wr, 99 op/s 2024-04-23T18:34:46.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:46 smithi172 ceph-mon[32456]: pgmap v449: 97 pgs: 97 active+clean; 745 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 17 MiB/s wr, 99 op/s 2024-04-23T18:34:46.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:46 smithi037 ceph-mon[29404]: pgmap v449: 97 pgs: 97 active+clean; 745 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 17 MiB/s wr, 99 op/s 2024-04-23T18:34:47.889 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T18:34:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T18:34:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 0, 2024-04-23T18:34:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 2, 2024-04-23T18:34:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 0, 2024-04-23T18:34:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 0, 2024-04-23T18:34:47.890 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 0, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.000990949, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.000221848, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:47.891 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T18:34:47.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 285, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 272, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1609520, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 166912, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 2, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 1, 2024-04-23T18:34:47.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.053209061, 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.015934947, 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.039454759, 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.006531240, 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 272, 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.013511433, 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000049674 2024-04-23T18:34:47.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1609520, 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 166912 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.895 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 206, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 320, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 192336, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 5, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.013164392, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.008963485, 2024-04-23T18:34:47.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.001304628, 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 203, 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.019122916, 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000094201 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:47.897 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 192336 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 0, 2024-04-23T18:34:47.898 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T18:34:47.899 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T18:34:47.900 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T18:34:47.901 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T18:34:47.902 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 148955, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 30, 2024-04-23T18:34:47.903 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 0, 2024-04-23T18:34:47.904 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 928, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 2, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T18:34:47.905 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 283, 2024-04-23T18:34:47.906 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1566877, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 283, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 283, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1566877, 2024-04-23T18:34:47.907 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T18:34:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T18:34:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T18:34:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T18:34:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:47.908 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T18:34:47.908 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T18:34:48.474 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:48 smithi173 ceph-mon[32906]: pgmap v450: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 107 op/s 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-23T18:34:48.500 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-23T18:34:48.501 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-23T18:34:48.515 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T18:34:48.515 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:34:48.516 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:34:48.537 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:34:48.537 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:34:48.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:48 smithi172 ceph-mon[32456]: pgmap v450: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 107 op/s 2024-04-23T18:34:48.599 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:34:48.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:48 smithi037 ceph-mon[29404]: pgmap v450: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 107 op/s 2024-04-23T18:34:48.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 -- 172.21.15.37:0/3580735020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec102bf0 msgr2=0x7fc6ec106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 --2- 172.21.15.37:0/3580735020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec102bf0 0x7fc6ec106ba0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:48.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 -- 172.21.15.37:0/3580735020 shutdown_connections 2024-04-23T18:34:48.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 --2- 172.21.15.37:0/3580735020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6ec107280 0x7fc6ec10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 --2- 172.21.15.37:0/3580735020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec102bf0 0x7fc6ec106ba0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 --2- 172.21.15.37:0/3580735020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec102240 0x7fc6ec102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 -- 172.21.15.37:0/3580735020 >> 172.21.15.37:0/3580735020 conn(0x7fc6ec0790e0 msgr2=0x7fc6ec0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 -- 172.21.15.37:0/3580735020 shutdown_connections 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 -- 172.21.15.37:0/3580735020 wait complete. 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 Processor -- start 2024-04-23T18:34:48.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.743+0000 7fc6f2620640 1 -- start start 2024-04-23T18:34:48.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f2620640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec102240 0x7fc6ec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f2620640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 0x7fc6ec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f2620640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6ec19d010 0x7fc6ec19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f2620640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec112210 con 0x7fc6ec107280 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f2620640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec112090 con 0x7fc6ec102240 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f2620640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec112390 con 0x7fc6ec19d010 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 0x7fc6ec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 0x7fc6ec198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33106/0 (socket says 172.21.15.37:33106) 2024-04-23T18:34:48.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 -- 172.21.15.37:0/2513587109 learned_addr learned my addr 172.21.15.37:0/2513587109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6ebfff640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec102240 0x7fc6ec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6f0bbd640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6ec19d010 0x7fc6ec19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 -- 172.21.15.37:0/2513587109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6ec19d010 msgr2=0x7fc6ec19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6ec19d010 0x7fc6ec19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 -- 172.21.15.37:0/2513587109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec102240 msgr2=0x7fc6ec198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec102240 0x7fc6ec198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 -- 172.21.15.37:0/2513587109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec19db00 con 0x7fc6ec107280 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6eb7fe640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 0x7fc6ec198c90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005ab0 tx=0x7fc6dc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.744+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc03f070 con 0x7fc6ec107280 2024-04-23T18:34:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.745+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc031b50 con 0x7fc6ec107280 2024-04-23T18:34:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.745+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec19dd90 con 0x7fc6ec107280 2024-04-23T18:34:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.745+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc035030 con 0x7fc6ec107280 2024-04-23T18:34:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.745+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec1a55b0 con 0x7fc6ec107280 2024-04-23T18:34:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.746+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec0691c0 con 0x7fc6ec107280 2024-04-23T18:34:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.747+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6dc042070 con 0x7fc6ec107280 2024-04-23T18:34:48.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.747+0000 7fc6e97fa640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3da0 0x7fc6d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:48.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.747+0000 7fc6ebfff640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3da0 0x7fc6d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:48.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.747+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fc6dc0f0d90 con 0x7fc6ec107280 2024-04-23T18:34:48.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.748+0000 7fc6ebfff640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3da0 0x7fc6d80a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40058c0 tx=0x7fc6d400a2d0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:48.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.752+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6dc0bcda0 con 0x7fc6ec107280 2024-04-23T18:34:48.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.942+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6ec102bf0 con 0x7fc6ec107280 2024-04-23T18:34:48.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.944+0000 7fc6e97fa640 1 -- 172.21.15.37:0/2513587109 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+23259 (secure 0 0 0) 0x7fc6dc0c10a0 con 0x7fc6ec107280 2024-04-23T18:34:48.936 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:48.936 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:34:11.037181+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:48.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3da0 msgr2=0x7fc6d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3da0 0x7fc6d80a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40058c0 tx=0x7fc6d400a2d0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 msgr2=0x7fc6ec198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 0x7fc6ec198c90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005ab0 tx=0x7fc6dc003b90 comp rx=0 tx=0).stop 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 shutdown_connections 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fc6d80a3da0 0x7fc6d80a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6ec19d010 0x7fc6ec19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6ec107280 0x7fc6ec198c90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 --2- 172.21.15.37:0/2513587109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec102240 0x7fc6ec198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.946+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 >> 172.21.15.37:0/2513587109 conn(0x7fc6ec0790e0 msgr2=0x7fc6ec100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.947+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 shutdown_connections 2024-04-23T18:34:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:48.947+0000 7fc6f2620640 1 -- 172.21.15.37:0/2513587109 wait complete. 2024-04-23T18:34:48.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:14:05.867833+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:18:32.516374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:18:33.348637+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:34:48.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:34:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 -- 172.21.15.37:0/658972881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc107270 msgr2=0x7f1dfc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 --2- 172.21.15.37:0/658972881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc107270 0x7f1dfc10ee40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009ca0 tx=0x7f1df402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 -- 172.21.15.37:0/658972881 shutdown_connections 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 --2- 172.21.15.37:0/658972881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc107270 0x7f1dfc10ee40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 --2- 172.21.15.37:0/658972881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc06b1f0 0x7f1dfc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 --2- 172.21.15.37:0/658972881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dfc06a840 0x7f1dfc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 -- 172.21.15.37:0/658972881 >> 172.21.15.37:0/658972881 conn(0x7f1dfc077eb0 msgr2=0x7f1dfc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 -- 172.21.15.37:0/658972881 shutdown_connections 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.065+0000 7f1e02720640 1 -- 172.21.15.37:0/658972881 wait complete. 2024-04-23T18:34:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 Processor -- start 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 -- start start 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dfc06a840 0x7f1dfc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc107270 0x7f1dfc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 0x7f1dfc19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc1121b0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112030 con 0x7f1dfc06a840 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e02720640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc112330 con 0x7f1dfc107270 2024-04-23T18:34:49.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e00cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 0x7f1dfc19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e00cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 0x7f1dfc19d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33132/0 (socket says 172.21.15.37:33132) 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e00cbd640 1 -- 172.21.15.37:0/521677692 learned_addr learned my addr 172.21.15.37:0/521677692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1dfb7fe640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc107270 0x7f1dfc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e00cbd640 1 -- 172.21.15.37:0/521677692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc107270 msgr2=0x7f1dfc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e00cbd640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc107270 0x7f1dfc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1dfbfff640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dfc06a840 0x7f1dfc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.066+0000 7f1e00cbd640 1 -- 172.21.15.37:0/521677692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dfc06a840 msgr2=0x7f1dfc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1e00cbd640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dfc06a840 0x7f1dfc1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1e00cbd640 1 -- 172.21.15.37:0/521677692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc19d9a0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1e00cbd640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 0x7f1dfc19d360 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005cc0 tx=0x7f1df402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4003cf0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df40044f0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc19dc30 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df402fd10 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.067+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1a5450 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.068+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc0691c0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.069+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1df4004040 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.070+0000 7f1df97fa640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1de40a3d20 0x7f1de40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.070+0000 7f1dfbfff640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1de40a3d20 0x7f1de40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.070+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f1df40f1fb0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.070+0000 7f1dfbfff640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1de40a3d20 0x7f1de40a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00ba40 tx=0x7f1dec005cc0 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.074+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1df40be720 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.264+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1dfc06b1f0 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.264+0000 7f1df97fa640 1 -- 172.21.15.37:0/521677692 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f1df40be020 con 0x7f1dfc19ceb0 2024-04-23T18:34:49.256 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-23T18:34:49.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1de40a3d20 msgr2=0x7f1de40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1de40a3d20 0x7f1de40a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00ba40 tx=0x7f1dec005cc0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 msgr2=0x7f1dfc19d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 0x7f1dfc19d360 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1df4005cc0 tx=0x7f1df402f470 comp rx=0 tx=0).stop 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 shutdown_connections 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f1de40a3d20 0x7f1de40a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1dfc19ceb0 0x7f1dfc19d360 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc107270 0x7f1dfc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 --2- 172.21.15.37:0/521677692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dfc06a840 0x7f1dfc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 >> 172.21.15.37:0/521677692 conn(0x7f1dfc077eb0 msgr2=0x7f1dfc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:49.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.266+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 shutdown_connections 2024-04-23T18:34:49.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.267+0000 7f1e02720640 1 -- 172.21.15.37:0/521677692 wait complete. 2024-04-23T18:34:49.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:49 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2513587109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:49.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:34:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 -- 172.21.15.37:0/1867669687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06a5b0 msgr2=0x7fcf9c06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/1867669687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06a5b0 0x7fcf9c06aa30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009930 tx=0x7fcf9002a420 comp rx=0 tx=0).stop 2024-04-23T18:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 -- 172.21.15.37:0/1867669687 shutdown_connections 2024-04-23T18:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/1867669687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c06af70 0x7fcf9c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/1867669687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06a5b0 0x7fcf9c06aa30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/1867669687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf9c108060 0x7fcf9c108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 -- 172.21.15.37:0/1867669687 >> 172.21.15.37:0/1867669687 conn(0x7fcf9c077c40 msgr2=0x7fcf9c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 -- 172.21.15.37:0/1867669687 shutdown_connections 2024-04-23T18:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.383+0000 7fcfa1f36640 1 -- 172.21.15.37:0/1867669687 wait complete. 2024-04-23T18:34:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 Processor -- start 2024-04-23T18:34:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 -- start start 2024-04-23T18:34:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf9c06a5b0 0x7fcf9c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 0x7fcf9c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108060 0x7fcf9c101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c112220 con 0x7fcf9c06af70 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c1120a0 con 0x7fcf9c108060 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcfa1f36640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c1123a0 con 0x7fcf9c06a5b0 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcf9affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 0x7fcf9c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcf9affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 0x7fcf9c1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33140/0 (socket says 172.21.15.37:33140) 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.384+0000 7fcf9affd640 1 -- 172.21.15.37:0/3750656737 learned_addr learned my addr 172.21.15.37:0/3750656737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9b7fe640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf9c06a5b0 0x7fcf9c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9bfff640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108060 0x7fcf9c101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9affd640 1 -- 172.21.15.37:0/3750656737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf9c06a5b0 msgr2=0x7fcf9c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9affd640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf9c06a5b0 0x7fcf9c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9affd640 1 -- 172.21.15.37:0/3750656737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108060 msgr2=0x7fcf9c101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9affd640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108060 0x7fcf9c101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9affd640 1 -- 172.21.15.37:0/3750656737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c1a6c20 con 0x7fcf9c06af70 2024-04-23T18:34:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9bfff640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108060 0x7fcf9c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf9affd640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 0x7fcf9c1016b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf90004770 tx=0x7fcf900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9003f070 con 0x7fcf9c06af70 2024-04-23T18:34:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf9003b460 con 0x7fcf9c06af70 2024-04-23T18:34:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c1a6eb0 con 0x7fcf9c06af70 2024-04-23T18:34:49.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90035070 con 0x7fcf9c06af70 2024-04-23T18:34:49.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.385+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c1a7310 con 0x7fcf9c06af70 2024-04-23T18:34:49.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.386+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c103650 con 0x7fcf9c06af70 2024-04-23T18:34:49.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.388+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf9003b8d0 con 0x7fcf9c06af70 2024-04-23T18:34:49.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.388+0000 7fcf98ff9640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcf840a3d20 0x7fcf840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.389+0000 7fcf9b7fe640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcf840a3d20 0x7fcf840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.389+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fcf900f0e40 con 0x7fcf9c06af70 2024-04-23T18:34:49.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.389+0000 7fcf9b7fe640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcf840a3d20 0x7fcf840a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c005fd0 tx=0x7fcf8c00b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:49.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.392+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf900bd4a0 con 0x7fcf9c06af70 2024-04-23T18:34:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.576+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf9c02d5c0 con 0x7fcf9c06af70 2024-04-23T18:34:49.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.577+0000 7fcf98ff9640 1 -- 172.21.15.37:0/3750656737 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fcf900bcda0 con 0x7fcf9c06af70 2024-04-23T18:34:49.569 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcf840a3d20 msgr2=0x7fcf840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcf840a3d20 0x7fcf840a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c005fd0 tx=0x7fcf8c00b390 comp rx=0 tx=0).stop 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 msgr2=0x7fcf9c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 0x7fcf9c1016b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf90004770 tx=0x7fcf900042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 shutdown_connections 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fcf840a3d20 0x7fcf840a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108060 0x7fcf9c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9c06af70 0x7fcf9c1016b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 --2- 172.21.15.37:0/3750656737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf9c06a5b0 0x7fcf9c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 >> 172.21.15.37:0/3750656737 conn(0x7fcf9c077c40 msgr2=0x7fcf9c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:49.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 shutdown_connections 2024-04-23T18:34:49.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.579+0000 7fcfa1f36640 1 -- 172.21.15.37:0/3750656737 wait complete. 2024-04-23T18:34:49.582 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:34:49.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:49 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2513587109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:49.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:49 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2513587109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:49.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3472059173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c101a40 msgr2=0x7f559c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3472059173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c101a40 0x7f559c105a80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f558c009900 tx=0x7f558c02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:49.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3472059173 shutdown_connections 2024-04-23T18:34:49.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3472059173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c106160 0x7f559c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3472059173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c101a40 0x7f559c105a80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3472059173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f559c101090 0x7f559c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3472059173 >> 172.21.15.37:0/3472059173 conn(0x7f559c077f50 msgr2=0x7f559c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:49.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3472059173 shutdown_connections 2024-04-23T18:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.696+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3472059173 wait complete. 2024-04-23T18:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 Processor -- start 2024-04-23T18:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 -- start start 2024-04-23T18:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c101090 0x7f559c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f559c106160 0x7f559c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 0x7f559c1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1110a0 con 0x7f559c1035a0 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f559c110f20 con 0x7f559c106160 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a3a6b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f559c111220 con 0x7f559c101090 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 0x7f559c1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 0x7f559c1a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33158/0 (socket says 172.21.15.37:33158) 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 -- 172.21.15.37:0/3781301623 learned_addr learned my addr 172.21.15.37:0/3781301623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a1807640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c101090 0x7f559c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a1006640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f559c106160 0x7f559c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 -- 172.21.15.37:0/3781301623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c101090 msgr2=0x7f559c102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c101090 0x7f559c102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 -- 172.21.15.37:0/3781301623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f559c106160 msgr2=0x7f559c103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f559c106160 0x7f559c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 -- 172.21.15.37:0/3781301623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c1a7010 con 0x7f559c1035a0 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a1807640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c101090 0x7f559c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.697+0000 7f55a2008640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 0x7f559c1a6910 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f559400eb10 tx=0x7f559400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.698+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594004b20 con 0x7f559c1035a0 2024-04-23T18:34:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.698+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5594004cc0 con 0x7f559c1035a0 2024-04-23T18:34:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.698+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c1a7300 con 0x7f559c1035a0 2024-04-23T18:34:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.698+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594010670 con 0x7f559c1035a0 2024-04-23T18:34:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.698+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1a78a0 con 0x7f559c1035a0 2024-04-23T18:34:49.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.699+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c068d70 con 0x7f559c1035a0 2024-04-23T18:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.700+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5594004e70 con 0x7f559c1035a0 2024-04-23T18:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.700+0000 7f5592ffd640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f55880a3dd0 0x7f55880a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.701+0000 7f55a1807640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f55880a3dd0 0x7f55880a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.701+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f55940d1010 con 0x7f559c1035a0 2024-04-23T18:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.701+0000 7f55a1807640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f55880a3dd0 0x7f55880a6290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f55980058c0 tx=0x7f5598005850 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.704+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5594053ff0 con 0x7f559c1035a0 2024-04-23T18:34:49.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.891+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f559c101a40 con 0x7f559c1035a0 2024-04-23T18:34:49.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.891+0000 7f5592ffd640 1 -- 172.21.15.37:0/3781301623 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f559409d610 con 0x7f559c1035a0 2024-04-23T18:34:49.884 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-23T18:34:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f55880a3dd0 msgr2=0x7f55880a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f55880a3dd0 0x7f55880a6290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f55980058c0 tx=0x7f5598005850 comp rx=0 tx=0).stop 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 msgr2=0x7f559c1a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 0x7f559c1a6910 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f559400eb10 tx=0x7f559400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 shutdown_connections 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f55880a3dd0 0x7f55880a6290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f559c1035a0 0x7f559c1a6910 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f559c106160 0x7f559c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 --2- 172.21.15.37:0/3781301623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559c101090 0x7f559c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.893+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 >> 172.21.15.37:0/3781301623 conn(0x7f559c077f50 msgr2=0x7f559c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.894+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 shutdown_connections 2024-04-23T18:34:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:49.894+0000 7f55a3a6b640 1 -- 172.21.15.37:0/3781301623 wait complete. 2024-04-23T18:34:49.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T18:34:50.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 -- 172.21.15.37:0/3280640154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101ac0 msgr2=0x7feb7c105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:50.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 --2- 172.21.15.37:0/3280640154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101ac0 0x7feb7c105ab0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7feb68009900 tx=0x7feb6802a410 comp rx=0 tx=0).stop 2024-04-23T18:34:50.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 -- 172.21.15.37:0/3280640154 shutdown_connections 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 --2- 172.21.15.37:0/3280640154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 0x7feb7c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 --2- 172.21.15.37:0/3280640154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101ac0 0x7feb7c105ab0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 --2- 172.21.15.37:0/3280640154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb7c101110 0x7feb7c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 -- 172.21.15.37:0/3280640154 >> 172.21.15.37:0/3280640154 conn(0x7feb7c077fb0 msgr2=0x7feb7c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 -- 172.21.15.37:0/3280640154 shutdown_connections 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.008+0000 7feb82c1e640 1 -- 172.21.15.37:0/3280640154 wait complete. 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 Processor -- start 2024-04-23T18:34:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 -- start start 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 0x7feb7c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 0x7feb7c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb7c19d010 0x7feb7c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c112210 con 0x7feb7c101110 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c112090 con 0x7feb7c106190 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb82c1e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c112390 con 0x7feb7c19d010 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 0x7feb7c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 0x7feb7c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33174/0 (socket says 172.21.15.37:33174) 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 -- 172.21.15.37:0/2748556109 learned_addr learned my addr 172.21.15.37:0/2748556109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb811bb640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb7c19d010 0x7feb7c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb7bfff640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 0x7feb7c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 -- 172.21.15.37:0/2748556109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb7c19d010 msgr2=0x7feb7c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb7c19d010 0x7feb7c19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 -- 172.21.15.37:0/2748556109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 msgr2=0x7feb7c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 0x7feb7c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb809ba640 1 -- 172.21.15.37:0/2748556109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c19db00 con 0x7feb7c101110 2024-04-23T18:34:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.009+0000 7feb7bfff640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 0x7feb7c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.010+0000 7feb809ba640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 0x7feb7c198750 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c890 tx=0x7feb7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.010+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70004220 con 0x7feb7c101110 2024-04-23T18:34:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.010+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb700043c0 con 0x7feb7c101110 2024-04-23T18:34:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.010+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c19ddf0 con 0x7feb7c101110 2024-04-23T18:34:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.010+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70004a00 con 0x7feb7c101110 2024-04-23T18:34:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.010+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c1a55e0 con 0x7feb7c101110 2024-04-23T18:34:50.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.011+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c0691c0 con 0x7feb7c101110 2024-04-23T18:34:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.012+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb700079b0 con 0x7feb7c101110 2024-04-23T18:34:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.012+0000 7feb79ffb640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb6c0a3d20 0x7feb6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.013+0000 7feb7bfff640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb6c0a3d20 0x7feb6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.013+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7feb700d05f0 con 0x7feb7c101110 2024-04-23T18:34:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.013+0000 7feb7bfff640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb6c0a3d20 0x7feb6c0a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feb68000c00 tx=0x7feb68031400 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.017+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb70014030 con 0x7feb7c101110 2024-04-23T18:34:50.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.216+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb7c199aa0 con 0x7feb7c101110 2024-04-23T18:34:50.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.217+0000 7feb79ffb640 1 -- 172.21.15.37:0/2748556109 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) ==== 81+0+943 (secure 0 0 0) 0x7feb7009cbf0 con 0x7feb7c101110 2024-04-23T18:34:50.209 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/3159991179 2024-04-24T18:20:30.552663+0000 2024-04-23T18:34:50.209 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/906783050 2024-04-24T18:20:30.552663+0000 2024-04-23T18:34:50.209 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2228885295 2024-04-24T18:20:30.552663+0000 2024-04-23T18:34:50.209 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3125793004 2024-04-24T18:11:39.779470+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2883572010 2024-04-24T18:20:30.552663+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/4000629980 2024-04-24T18:11:39.779470+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2775309719 2024-04-24T18:20:30.552663+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3421057704 2024-04-24T18:11:39.779470+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1821395295 2024-04-24T18:11:13.315493+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1257180359 2024-04-24T18:11:13.315493+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/4000629980 2024-04-24T18:11:39.779470+0000 2024-04-23T18:34:50.210 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/78121481 2024-04-24T18:11:13.315493+0000 2024-04-23T18:34:50.211 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/3159991179 2024-04-24T18:20:30.552663+0000 2024-04-23T18:34:50.211 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1641631132 2024-04-24T18:11:13.315493+0000 2024-04-23T18:34:50.211 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1641631132 2024-04-24T18:11:13.315493+0000 2024-04-23T18:34:50.211 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4219584537 2024-04-24T18:11:39.779470+0000 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb6c0a3d20 msgr2=0x7feb6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb6c0a3d20 0x7feb6c0a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feb68000c00 tx=0x7feb68031400 comp rx=0 tx=0).stop 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 msgr2=0x7feb7c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 0x7feb7c198750 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c890 tx=0x7feb7000cd50 comp rx=0 tx=0).stop 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 shutdown_connections 2024-04-23T18:34:50.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7feb6c0a3d20 0x7feb6c0a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb7c19d010 0x7feb7c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb7c106190 0x7feb7c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 --2- 172.21.15.37:0/2748556109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb7c101110 0x7feb7c198750 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 >> 172.21.15.37:0/2748556109 conn(0x7feb7c077fb0 msgr2=0x7feb7c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 shutdown_connections 2024-04-23T18:34:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:50.219+0000 7feb82c1e640 1 -- 172.21.15.37:0/2748556109 wait complete. 2024-04-23T18:34:50.222 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-23T18:34:50.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:50.222 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-23T18:34:50.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:50 smithi172 ceph-mon[32456]: pgmap v451: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2024-04-23T18:34:50.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:50 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/521677692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:50 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3750656737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:50 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/3781301623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:50 smithi173 ceph-mon[32906]: pgmap v451: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2024-04-23T18:34:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:50 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/521677692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:50 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3750656737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:50 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/3781301623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:50 smithi037 ceph-mon[29404]: pgmap v451: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2024-04-23T18:34:50.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:50 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/521677692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:50 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3750656737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:50 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/3781301623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:34:50.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:34:50.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:50.721 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T18:34:50.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:50.780 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-23T18:34:50.796 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T18:34:50.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:50.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:34:50.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:50.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-23T18:34:50.958 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T18:34:50.958 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-23T18:34:50.958 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:34:50.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056880,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498928k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16609 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249460k,nr_inodes=812365,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b57cc6bace4be40389159614d745199062128462183a0b3c41cbc05cac776bd1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/b57cc6bace4be40389159614d745199062128462183a0b3c41cbc05cac776bd1/diff,workdir=/var/lib/containers/storage/overlay/b57cc6bace4be40389159614d745199062128462183a0b3c41cbc05cac776bd1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f507d673d6ac35c45389ed0d1edf7f328ec0ed2bf6a434abc6e2477bd46278f7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/f507d673d6ac35c45389ed0d1edf7f328ec0ed2bf6a434abc6e2477bd46278f7/diff,workdir=/var/lib/containers/storage/overlay/f507d673d6ac35c45389ed0d1edf7f328ec0ed2bf6a434abc6e2477bd46278f7/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249460k,nr_inodes=812365,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/b96c99a4780e91cc8ab9eae95b2da12725ae49e08f8b1288f0199cc3ca205399/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/b96c99a4780e91cc8ab9eae95b2da12725ae49e08f8b1288f0199cc3ca205399/diff,workdir=/var/lib/containers/storage/overlay/b96c99a4780e91cc8ab9eae95b2da12725ae49e08f8b1288f0199cc3ca205399/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/329cb720dc3adbe73ccab5b8c75ba56278e09e36c58866fbeb4ff456d594788b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/329cb720dc3adbe73ccab5b8c75ba56278e09e36c58866fbeb4ff456d594788b/diff,workdir=/var/lib/containers/storage/overlay/329cb720dc3adbe73ccab5b8c75ba56278e09e36c58866fbeb4ff456d594788b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/131a27809f075d8b5d98c0f08eae352558e0c0cf519288b0fd4152ce3961038a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/131a27809f075d8b5d98c0f08eae352558e0c0cf519288b0fd4152ce3961038a/diff,workdir=/var/lib/containers/storage/overlay/131a27809f075d8b5d98c0f08eae352558e0c0cf519288b0fd4152ce3961038a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c66bd01f0f3fb77633645f2768c8f421f1703dd6f124bf6be6153a567e6dde60/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/c66bd01f0f3fb77633645f2768c8f421f1703dd6f124bf6be6153a567e6dde60/diff,workdir=/var/lib/containers/storage/overlay/c66bd01f0f3fb77633645f2768c8f421f1703dd6f124bf6be6153a567e6dde60/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2b2206cbbd98159a57b7adc7434096443c45b039cccbcb4a3003effe3d1da3da/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/2b2206cbbd98159a57b7adc7434096443c45b039cccbcb4a3003effe3d1da3da/diff,workdir=/var/lib/containers/storage/overlay/2b2206cbbd98159a57b7adc7434096443c45b039cccbcb4a3003effe3d1da3da/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0bf237c15ed04c4a001c8371af5532bfd1277677b2103686918aade5cea26f74/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/0bf237c15ed04c4a001c8371af5532bfd1277677b2103686918aade5cea26f74/diff,workdir=/var/lib/containers/storage/overlay/0bf237c15ed04c4a001c8371af5532bfd1277677b2103686918aade5cea26f74/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/07d89045278f93c9ea87f21efd3f8e177b72731bc56f553fe9fecba2fdbcfb5e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/07d89045278f93c9ea87f21efd3f8e177b72731bc56f553fe9fecba2fdbcfb5e/diff,workdir=/var/lib/containers/storage/overlay/07d89045278f93c9ea87f21efd3f8e177b72731bc56f553fe9fecba2fdbcfb5e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f255d076aca99d46b9c5e5ef3469a9bd122ddc68ce5c8ee6aa1024a737b12561/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/BRVUTNMXIEFN6K4KOWFJLW2VWW:/var/lib/containers/storage/overlay/l/6QWJSMO3YRZP7KU3BCAWTO35QU,upperdir=/var/lib/containers/storage/overlay/f255d076aca99d46b9c5e5ef3469a9bd122ddc68ce5c8ee6aa1024a737b12561/diff,workdir=/var/lib/containers/storage/overlay/f255d076aca99d46b9c5e5ef3469a9bd122ddc68ce5c8ee6aa1024a737b12561/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498928k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-23T18:34:50.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:50.985 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-23T18:34:51.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:34:51.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-23T18:34:51.108 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-23T18:34:51.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:34:51.110 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T18:34:51.121 INFO:tasks.cephadm:Teardown begin 2024-04-23T18:34:51.122 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T18:34:51.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 -- 172.21.15.37:0/2895467268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 msgr2=0x7f63381035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 --2- 172.21.15.37:0/2895467268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f63381035c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6328009900 tx=0x7f632802a510 comp rx=0 tx=0).stop 2024-04-23T18:34:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 -- 172.21.15.37:0/2895467268 shutdown_connections 2024-04-23T18:34:51.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 --2- 172.21.15.37:0/2895467268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633810c520 0x7f633810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 --2- 172.21.15.37:0/2895467268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6338103b00 0x7f633810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 --2- 172.21.15.37:0/2895467268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f63381035c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 -- 172.21.15.37:0/2895467268 >> 172.21.15.37:0/2895467268 conn(0x7f6338077f50 msgr2=0x7f6338078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:51.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 -- 172.21.15.37:0/2895467268 shutdown_connections 2024-04-23T18:34:51.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.271+0000 7f633d3ad640 1 -- 172.21.15.37:0/2895467268 wait complete. 2024-04-23T18:34:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 Processor -- start 2024-04-23T18:34:51.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 -- start start 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f633819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6338103b00 0x7f633819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633810c520 0x7f6338198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63381121b0 con 0x7f63381031e0 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6338112030 con 0x7f633810c520 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f633d3ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6338112330 con 0x7f6338103b00 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f632effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f633819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f632effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f633819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33186/0 (socket says 172.21.15.37:33186) 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f632effd640 1 -- 172.21.15.37:0/1975451597 learned_addr learned my addr 172.21.15.37:0/1975451597 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6338103b00 0x7f633819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.272+0000 7f63377fe640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633810c520 0x7f6338198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f632effd640 1 -- 172.21.15.37:0/1975451597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6338103b00 msgr2=0x7f633819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f632effd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6338103b00 0x7f633819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f632effd640 1 -- 172.21.15.37:0/1975451597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633810c520 msgr2=0x7f6338198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f632effd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633810c520 0x7f6338198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f632effd640 1 -- 172.21.15.37:0/1975451597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6338198d10 con 0x7f63381031e0 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6338103b00 0x7f633819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f63377fe640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633810c520 0x7f6338198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f632effd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f633819e3e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6328005940 tx=0x7f63280040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632803f030 con 0x7f63381031e0 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63280044c0 con 0x7f63381031e0 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328035070 con 0x7f63381031e0 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6338198fa0 con 0x7f63381031e0 2024-04-23T18:34:51.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.273+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63381a52a0 con 0x7f63381031e0 2024-04-23T18:34:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.274+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6338068d70 con 0x7f63381031e0 2024-04-23T18:34:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.276+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6328042030 con 0x7f63381031e0 2024-04-23T18:34:51.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.276+0000 7f6334ff9640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f63240a3da0 0x7f63240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.276+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f63240a3da0 0x7f63240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.276+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f63280f0d50 con 0x7f63381031e0 2024-04-23T18:34:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.277+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f63240a3da0 0x7f63240a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f633810e470 tx=0x7f6320005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:51.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.281+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63280bd350 con 0x7f63381031e0 2024-04-23T18:34:51.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:51 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2748556109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:34:51.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:51 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:51.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.510+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63381076a0 con 0x7f63381031e0 2024-04-23T18:34:51.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.511+0000 7f6334ff9640 1 -- 172.21.15.37:0/1975451597 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11880 (secure 0 0 0) 0x7f63280bcc50 con 0x7f63381031e0 2024-04-23T18:34:51.503 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:51.503 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:34:22.577713+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:34:51.504 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 50 2024-04-23T18:34:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f63240a3da0 msgr2=0x7f63240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f63240a3da0 0x7f63240a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f633810e470 tx=0x7f6320005f90 comp rx=0 tx=0).stop 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 msgr2=0x7f633819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f633819e3e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6328005940 tx=0x7f63280040c0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 shutdown_connections 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f63240a3da0 0x7f63240a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633810c520 0x7f6338198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6338103b00 0x7f633819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 --2- 172.21.15.37:0/1975451597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63381031e0 0x7f633819e3e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 >> 172.21.15.37:0/1975451597 conn(0x7f6338077f50 msgr2=0x7f63380fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:51.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 shutdown_connections 2024-04-23T18:34:51.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.513+0000 7f633d3ad640 1 -- 172.21.15.37:0/1975451597 wait complete. 2024-04-23T18:34:51.517 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:51.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:51 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2748556109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:34:51.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:51 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:51.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:51 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2748556109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T18:34:51.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:51 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 -- 172.21.15.37:0/901487644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe650101a50 msgr2=0x7fe650105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 --2- 172.21.15.37:0/901487644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe650101a50 0x7fe650105a90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009900 tx=0x7fe64c02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 -- 172.21.15.37:0/901487644 shutdown_connections 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 --2- 172.21.15.37:0/901487644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe650106170 0x7fe65010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 --2- 172.21.15.37:0/901487644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe650101a50 0x7fe650105a90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 --2- 172.21.15.37:0/901487644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6501010a0 0x7fe650101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 -- 172.21.15.37:0/901487644 >> 172.21.15.37:0/901487644 conn(0x7fe650077f50 msgr2=0x7fe650078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 -- 172.21.15.37:0/901487644 shutdown_connections 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.627+0000 7fe6586b9640 1 -- 172.21.15.37:0/901487644 wait complete. 2024-04-23T18:34:51.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 Processor -- start 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 -- start start 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 0x7fe650197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe650106170 0x7fe650197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe65019bf30 0x7fe65019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501110b0 con 0x7fe6501010a0 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe650110f30 con 0x7fe65019bf30 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe6586b9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe650111230 con 0x7fe650106170 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 0x7fe650197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 0x7fe650197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33194/0 (socket says 172.21.15.37:33194) 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 -- 172.21.15.37:0/404265451 learned_addr learned my addr 172.21.15.37:0/404265451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656c56640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe65019bf30 0x7fe65019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe655c54640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe650106170 0x7fe650197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 -- 172.21.15.37:0/404265451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe650106170 msgr2=0x7fe650197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe650106170 0x7fe650197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 -- 172.21.15.37:0/404265451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe65019bf30 msgr2=0x7fe65019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe65019bf30 0x7fe65019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 -- 172.21.15.37:0/404265451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65019ca20 con 0x7fe6501010a0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656c56640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe65019bf30 0x7fe65019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.628+0000 7fe656455640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 0x7fe650197710 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe64000d770 tx=0x7fe64000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.629+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640017070 con 0x7fe6501010a0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.629+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6400054d0 con 0x7fe6501010a0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.629+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65019cd10 con 0x7fe6501010a0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.629+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640004c30 con 0x7fe6501010a0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.629+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6501a4480 con 0x7fe6501010a0 2024-04-23T18:34:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.630+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650068d70 con 0x7fe6501010a0 2024-04-23T18:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.631+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe640004dd0 con 0x7fe6501010a0 2024-04-23T18:34:51.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.631+0000 7fe6477fe640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.632+0000 7fe655c54640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.632+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fe6400d07b0 con 0x7fe6501010a0 2024-04-23T18:34:51.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.632+0000 7fe655c54640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe64c000c00 tx=0x7fe64c031400 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:51.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.635+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe64009cdb0 con 0x7fe6501010a0 2024-04-23T18:34:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.865+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6501989c0 con 0x7fe6501010a0 2024-04-23T18:34:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.866+0000 7fe6477fe640 1 -- 172.21.15.37:0/404265451 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11880 (secure 0 0 0) 0x7fe6501989c0 con 0x7fe6501010a0 2024-04-23T18:34:51.858 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:51.858 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3749554731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3749554731},{"type":"v1","addr":"172.21.15.37:6835","nonce":3749554731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3961977132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3961977132},{"type":"v1","addr":"172.21.15.37:6837","nonce":3961977132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/514853653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":514853653},{"type":"v1","addr":"172.21.15.37:6839","nonce":514853653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2155547418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2155547418},{"type":"v1","addr":"172.21.15.37:6841","nonce":2155547418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3950527293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3950527293},{"type":"v1","addr":"172.21.15.172:6833","nonce":3950527293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2179419272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2179419272},{"type":"v1","addr":"172.21.15.173:6833","nonce":2179419272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:18:34.408115+0000","modified":"2024-04-23T18:34:22.577713+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2640036963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2640036963},{"type":"v1","addr":"172.21.15.172:6835","nonce":2640036963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2419231317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2419231317},{"type":"v1","addr":"172.21.15.173:6835","nonce":2419231317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/364708999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":364708999},{"type":"v1","addr":"172.21.15.172:6837","nonce":364708999}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.173:6839/496716250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":496716250},{"type":"v1","addr":"172.21.15.173:6839","nonce":496716250}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6839/2128854391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":2128854391},{"type":"v1","addr":"172.21.15.172:6839","nonce":2128854391}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25606":{"gid":25606,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6837/3140945410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3140945410},{"type":"v1","addr":"172.21.15.173:6837","nonce":3140945410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24479,24484]},"id":1}]} 2024-04-23T18:34:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 50 2024-04-23T18:34:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe63c0a3d20 msgr2=0x7fe63c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe64c000c00 tx=0x7fe64c031400 comp rx=0 tx=0).stop 2024-04-23T18:34:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 msgr2=0x7fe650197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 0x7fe650197710 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe64000d770 tx=0x7fe64000dc30 comp rx=0 tx=0).stop 2024-04-23T18:34:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 shutdown_connections 2024-04-23T18:34:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fe63c0a3d20 0x7fe63c0a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe65019bf30 0x7fe65019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe650106170 0x7fe650197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 --2- 172.21.15.37:0/404265451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6501010a0 0x7fe650197710 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 >> 172.21.15.37:0/404265451 conn(0x7fe650077f50 msgr2=0x7fe6500ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 shutdown_connections 2024-04-23T18:34:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.868+0000 7fe6586b9640 1 -- 172.21.15.37:0/404265451 wait complete. 2024-04-23T18:34:51.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:34:51.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 -- 172.21.15.37:0/1771434690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32081010a0 msgr2=0x7f3208101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 --2- 172.21.15.37:0/1771434690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32081010a0 0x7f3208101480 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009900 tx=0x7f31f802a410 comp rx=0 tx=0).stop 2024-04-23T18:34:51.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 -- 172.21.15.37:0/1771434690 shutdown_connections 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 --2- 172.21.15.37:0/1771434690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3208106170 0x7f320810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 --2- 172.21.15.37:0/1771434690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3208101a50 0x7f3208105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 --2- 172.21.15.37:0/1771434690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32081010a0 0x7f3208101480 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 -- 172.21.15.37:0/1771434690 >> 172.21.15.37:0/1771434690 conn(0x7f3208077f50 msgr2=0x7f3208078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 -- 172.21.15.37:0/1771434690 shutdown_connections 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 -- 172.21.15.37:0/1771434690 wait complete. 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 Processor -- start 2024-04-23T18:34:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.981+0000 7f320e5e8640 1 -- start start 2024-04-23T18:34:51.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320e5e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32081010a0 0x7f3208196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320e5e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 0x7f3208196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320e5e8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320819adf0 0x7f320819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320e5e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f320810ffd0 con 0x7f3208106170 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320e5e8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f320810fe50 con 0x7f32081010a0 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320e5e8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3208110150 con 0x7f320819adf0 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 0x7f3208196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 0x7f3208196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33218/0 (socket says 172.21.15.37:33218) 2024-04-23T18:34:51.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 -- 172.21.15.37:0/2520497612 learned_addr learned my addr 172.21.15.37:0/2520497612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320cb85640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320819adf0 0x7f320819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f3207fff640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32081010a0 0x7f3208196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 -- 172.21.15.37:0/2520497612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320819adf0 msgr2=0x7f320819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320819adf0 0x7f320819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 -- 172.21.15.37:0/2520497612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32081010a0 msgr2=0x7f3208196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32081010a0 0x7f3208196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f32077fe640 1 -- 172.21.15.37:0/2520497612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320819b8e0 con 0x7f3208106170 2024-04-23T18:34:51.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f320cb85640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320819adf0 0x7f320819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.982+0000 7f3207fff640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32081010a0 0x7f3208196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.983+0000 7f32077fe640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 0x7f3208196a70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005fd0 tx=0x7f31fc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.983+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc017070 con 0x7f3208106170 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.983+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320819bbd0 con 0x7f3208106170 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.983+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc009c30 con 0x7f3208106170 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.983+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc003d30 con 0x7f3208106170 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.983+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081a3410 con 0x7f3208106170 2024-04-23T18:34:51.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.984+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3208068d70 con 0x7f3208106170 2024-04-23T18:34:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.985+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31fc003ed0 con 0x7f3208106170 2024-04-23T18:34:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.985+0000 7f32057fa640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:51.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.986+0000 7f3207fff640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:51.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.986+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f31fc0d0cc0 con 0x7f3208106170 2024-04-23T18:34:51.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.986+0000 7f3207fff640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f31f00a3d20 0x7f31f00a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31f8031440 tx=0x7f31f8005a20 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:51.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:51.990+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31fc020030 con 0x7f3208106170 2024-04-23T18:34:52.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.178+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3208197880 con 0x7f3208106170 2024-04-23T18:34:52.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.180+0000 7f32057fa640 1 -- 172.21.15.37:0/2520497612 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+23259 (secure 0 0 0) 0x7f3208197880 con 0x7f3208106170 2024-04-23T18:34:52.172 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-23T18:34:52.172 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"af48a038-019c-11ef-bc93-c7b262605968","created":"2024-04-23T18:10:40.839640+0000","modified":"2024-04-23T18:34:11.037181+0000","last_up_change":"2024-04-23T18:17:47.796878+0000","last_in_change":"2024-04-23T18:17:27.824421+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:14:05.867833+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:18:32.516374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:18:33.348637+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d65e5f77-8366-4dbf-92bf-a34c53c514c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6803","nonce":3727253948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6805","nonce":3727253948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6809","nonce":3727253948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3727253948},{"type":"v1","addr":"172.21.15.37:6807","nonce":3727253948}]},"public_addr":"172.21.15.37:6803/3727253948","cluster_addr":"172.21.15.37:6805/3727253948","heartbeat_back_addr":"172.21.15.37:6809/3727253948","heartbeat_front_addr":"172.21.15.37:6807/3727253948","state":["exists","up"]},{"osd":1,"uuid":"28317ec5-ba88-452e-95ba-a312941588bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6801","nonce":1638984815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6803","nonce":1638984815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6807","nonce":1638984815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":1638984815},{"type":"v1","addr":"172.21.15.172:6805","nonce":1638984815}]},"public_addr":"172.21.15.172:6801/1638984815","cluster_addr":"172.21.15.172:6803/1638984815","heartbeat_back_addr":"172.21.15.172:6807/1638984815","heartbeat_front_addr":"172.21.15.172:6805/1638984815","state":["exists","up"]},{"osd":2,"uuid":"00f910a1-90d1-485c-a253-d0028ce0cc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6801","nonce":4145857941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6803","nonce":4145857941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6807","nonce":4145857941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":4145857941},{"type":"v1","addr":"172.21.15.173:6805","nonce":4145857941}]},"public_addr":"172.21.15.173:6801/4145857941","cluster_addr":"172.21.15.173:6803/4145857941","heartbeat_back_addr":"172.21.15.173:6807/4145857941","heartbeat_front_addr":"172.21.15.173:6805/4145857941","state":["exists","up"]},{"osd":3,"uuid":"6533c224-ca86-4499-9182-283a32bb2774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6811","nonce":1815509049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6813","nonce":1815509049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6817","nonce":1815509049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1815509049},{"type":"v1","addr":"172.21.15.37:6815","nonce":1815509049}]},"public_addr":"172.21.15.37:6811/1815509049","cluster_addr":"172.21.15.37:6813/1815509049","heartbeat_back_addr":"172.21.15.37:6817/1815509049","heartbeat_front_addr":"172.21.15.37:6815/1815509049","state":["exists","up"]},{"osd":4,"uuid":"1e24984b-edbd-4d4f-8716-6713ce2a039c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6809","nonce":3887503569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6811","nonce":3887503569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6815","nonce":3887503569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3887503569},{"type":"v1","addr":"172.21.15.172:6813","nonce":3887503569}]},"public_addr":"172.21.15.172:6809/3887503569","cluster_addr":"172.21.15.172:6811/3887503569","heartbeat_back_addr":"172.21.15.172:6815/3887503569","heartbeat_front_addr":"172.21.15.172:6813/3887503569","state":["exists","up"]},{"osd":5,"uuid":"c0152f97-2416-42e9-b0a0-6a7389cb0c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6809","nonce":1194532318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6811","nonce":1194532318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6815","nonce":1194532318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1194532318},{"type":"v1","addr":"172.21.15.173:6813","nonce":1194532318}]},"public_addr":"172.21.15.173:6809/1194532318","cluster_addr":"172.21.15.173:6811/1194532318","heartbeat_back_addr":"172.21.15.173:6815/1194532318","heartbeat_front_addr":"172.21.15.173:6813/1194532318","state":["exists","up"]},{"osd":6,"uuid":"5bc899de-4661-4c4f-b990-07db5cf3bd6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6819","nonce":825947067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6821","nonce":825947067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6825","nonce":825947067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":825947067},{"type":"v1","addr":"172.21.15.37:6823","nonce":825947067}]},"public_addr":"172.21.15.37:6819/825947067","cluster_addr":"172.21.15.37:6821/825947067","heartbeat_back_addr":"172.21.15.37:6825/825947067","heartbeat_front_addr":"172.21.15.37:6823/825947067","state":["exists","up"]},{"osd":7,"uuid":"1fb4b1b9-9fb4-475f-a40c-79ff6ba58324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6817","nonce":3090366504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6819","nonce":3090366504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6823","nonce":3090366504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3090366504},{"type":"v1","addr":"172.21.15.172:6821","nonce":3090366504}]},"public_addr":"172.21.15.172:6817/3090366504","cluster_addr":"172.21.15.172:6819/3090366504","heartbeat_back_addr":"172.21.15.172:6823/3090366504","heartbeat_front_addr":"172.21.15.172:6821/3090366504","state":["exists","up"]},{"osd":8,"uuid":"ec2a2669-fc68-4265-b68b-be4f008e4c2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969719555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6819","nonce":1969719555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6823","nonce":1969719555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1969719555},{"type":"v1","addr":"172.21.15.173:6821","nonce":1969719555}]},"public_addr":"172.21.15.173:6817/1969719555","cluster_addr":"172.21.15.173:6819/1969719555","heartbeat_back_addr":"172.21.15.173:6823/1969719555","heartbeat_front_addr":"172.21.15.173:6821/1969719555","state":["exists","up"]},{"osd":9,"uuid":"7eddb6e9-0b34-4a18-961f-0134670a2581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6827","nonce":1483571052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6829","nonce":1483571052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6833","nonce":1483571052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1483571052},{"type":"v1","addr":"172.21.15.37:6831","nonce":1483571052}]},"public_addr":"172.21.15.37:6827/1483571052","cluster_addr":"172.21.15.37:6829/1483571052","heartbeat_back_addr":"172.21.15.37:6833/1483571052","heartbeat_front_addr":"172.21.15.37:6831/1483571052","state":["exists","up"]},{"osd":10,"uuid":"72e96128-329b-4155-83cf-58bb44169b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6825","nonce":1398661350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6827","nonce":1398661350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6831","nonce":1398661350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1398661350},{"type":"v1","addr":"172.21.15.172:6829","nonce":1398661350}]},"public_addr":"172.21.15.172:6825/1398661350","cluster_addr":"172.21.15.172:6827/1398661350","heartbeat_back_addr":"172.21.15.172:6831/1398661350","heartbeat_front_addr":"172.21.15.172:6829/1398661350","state":["exists","up"]},{"osd":11,"uuid":"3d918711-72d6-4a04-97cc-4fe8dad4a3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6825","nonce":1559500794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6827","nonce":1559500794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6831","nonce":1559500794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1559500794},{"type":"v1","addr":"172.21.15.173:6829","nonce":1559500794}]},"public_addr":"172.21.15.173:6825/1559500794","cluster_addr":"172.21.15.173:6827/1559500794","heartbeat_back_addr":"172.21.15.173:6831/1559500794","heartbeat_front_addr":"172.21.15.173:6829/1559500794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:14.491530+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:13:38.339665+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:02.500874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:27.714616+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:14:52.106065+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:16.428880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:15:41.425824+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:05.738939+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:30.553911+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:16:56.134986+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:20.842230+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:17:45.791989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":6}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9},{"from":5,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/906783050":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/2228885295":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3125793004":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2883572010":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6801/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/2775309719":"2024-04-24T18:20:30.552663+0000","172.21.15.37:0/3421057704":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/1821395295":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/1257180359":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/4000629980":"2024-04-24T18:11:39.779470+0000","172.21.15.37:0/78121481":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6800/3159991179":"2024-04-24T18:20:30.552663+0000","172.21.15.37:6800/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:6801/1641631132":"2024-04-24T18:11:13.315493+0000","172.21.15.37:0/4219584537":"2024-04-24T18:11:39.779470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:34:52.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f31f00a3d20 msgr2=0x7f31f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f31f00a3d20 0x7f31f00a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31f8031440 tx=0x7f31f8005a20 comp rx=0 tx=0).stop 2024-04-23T18:34:52.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 msgr2=0x7f3208196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 0x7f3208196a70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005fd0 tx=0x7f31fc00dde0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 shutdown_connections 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320819adf0 0x7f320819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3208106170 0x7f3208196a70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 --2- 172.21.15.37:0/2520497612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32081010a0 0x7f3208196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 >> 172.21.15.37:0/2520497612 conn(0x7f3208077f50 msgr2=0x7f32080ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 shutdown_connections 2024-04-23T18:34:52.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.182+0000 7f320e5e8640 1 -- 172.21.15.37:0/2520497612 wait complete. 2024-04-23T18:34:52.185 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T18:34:52.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 -- 172.21.15.37:0/3067810517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 msgr2=0x7f566c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 --2- 172.21.15.37:0/3067810517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c101510 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f565c009900 tx=0x7f565c02a410 comp rx=0 tx=0).stop 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 -- 172.21.15.37:0/3067810517 shutdown_connections 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 --2- 172.21.15.37:0/3067810517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f566c106200 0x7f566c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 --2- 172.21.15.37:0/3067810517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c101ae0 0x7f566c105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 --2- 172.21.15.37:0/3067810517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c101510 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 -- 172.21.15.37:0/3067810517 >> 172.21.15.37:0/3067810517 conn(0x7f566c077f80 msgr2=0x7f566c078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 -- 172.21.15.37:0/3067810517 shutdown_connections 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.297+0000 7f5671d4f640 1 -- 172.21.15.37:0/3067810517 wait complete. 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 Processor -- start 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 -- start start 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c106200 0x7f566c103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f566c103640 0x7f566c1a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f566c111160 con 0x7f566c101130 2024-04-23T18:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f566c110fe0 con 0x7f566c103640 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f5671d4f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1112e0 con 0x7f566c106200 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.298+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33236/0 (socket says 172.21.15.37:33236) 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 -- 172.21.15.37:0/1331194653 learned_addr learned my addr 172.21.15.37:0/1331194653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566affd640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c106200 0x7f566c103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566bfff640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f566c103640 0x7f566c1a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 -- 172.21.15.37:0/1331194653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c106200 msgr2=0x7f566c103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c106200 0x7f566c103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 -- 172.21.15.37:0/1331194653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f566c103640 msgr2=0x7f566c1a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f566c103640 0x7f566c1a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 -- 172.21.15.37:0/1331194653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c1a7080 con 0x7f566c101130 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f566b7fe640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c102bc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f565c005940 tx=0x7f565c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c03f030 con 0x7f566c101130 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c003cb0 con 0x7f566c101130 2024-04-23T18:34:52.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c1a7310 con 0x7f566c101130 2024-04-23T18:34:52.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c035070 con 0x7f566c101130 2024-04-23T18:34:52.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.299+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a7800 con 0x7f566c101130 2024-04-23T18:34:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.300+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c102600 con 0x7f566c101130 2024-04-23T18:34:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.301+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f565c042030 con 0x7f566c101130 2024-04-23T18:34:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.302+0000 7f5668ff9640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f56540a3d20 0x7f56540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.302+0000 7f566affd640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f56540a3d20 0x7f56540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.302+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f565c0f10c0 con 0x7f566c101130 2024-04-23T18:34:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.302+0000 7f566affd640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f56540a3d20 0x7f56540a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f566c104600 tx=0x7f566000b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:52.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.306+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f565c0bd720 con 0x7f566c101130 2024-04-23T18:34:52.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.533+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f566c10a440 con 0x7f566c101130 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: pgmap v452: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1975451597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/404265451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2520497612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.537+0000 7f5668ff9640 1 -- 172.21.15.37:0/1331194653 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7f565c0bd020 con 0x7f566c101130 2024-04-23T18:34:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T18:34:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f56540a3d20 msgr2=0x7f56540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f56540a3d20 0x7f56540a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f566c104600 tx=0x7f566000b390 comp rx=0 tx=0).stop 2024-04-23T18:34:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 msgr2=0x7f566c102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c102bc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f565c005940 tx=0x7f565c0042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 shutdown_connections 2024-04-23T18:34:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f56540a3d20 0x7f56540a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f566c103640 0x7f566c1a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f566c106200 0x7f566c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 --2- 172.21.15.37:0/1331194653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566c101130 0x7f566c102bc0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 >> 172.21.15.37:0/1331194653 conn(0x7f566c077f80 msgr2=0x7f566c06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 shutdown_connections 2024-04-23T18:34:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.540+0000 7f5671d4f640 1 -- 172.21.15.37:0/1331194653 wait complete. 2024-04-23T18:34:52.543 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T18:34:52.575 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: pgmap v452: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2024-04-23T18:34:52.575 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:52.575 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:52.576 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:34:52.576 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1975451597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.576 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/404265451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.576 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2520497612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: pgmap v452: 97 pgs: 97 active+clean; 746 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.5 MiB/s wr, 58 op/s 2024-04-23T18:34:52.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:52.577 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:52.577 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' 2024-04-23T18:34:52.577 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1975451597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.577 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/404265451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.577 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2520497612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:34:52.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 -- 172.21.15.37:0/955125765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901014a0 msgr2=0x7fa590106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 --2- 172.21.15.37:0/955125765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901014a0 0x7fa590106bd0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa584009900 tx=0x7fa58402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 -- 172.21.15.37:0/955125765 shutdown_connections 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 --2- 172.21.15.37:0/955125765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5901072b0 0x7fa59010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 --2- 172.21.15.37:0/955125765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901014a0 0x7fa590106bd0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 --2- 172.21.15.37:0/955125765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 0x7fa590100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 -- 172.21.15.37:0/955125765 >> 172.21.15.37:0/955125765 conn(0x7fa5900760e0 msgr2=0x7fa5900764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 -- 172.21.15.37:0/955125765 shutdown_connections 2024-04-23T18:34:52.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.658+0000 7fa5951f7640 1 -- 172.21.15.37:0/955125765 wait complete. 2024-04-23T18:34:52.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 Processor -- start 2024-04-23T18:34:52.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 -- start start 2024-04-23T18:34:52.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 0x7fa59019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 0x7fa59019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa590198610 0x7fa590198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa590112230 con 0x7fa5901072b0 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901120b0 con 0x7fa590198610 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa5951f7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901123b0 con 0x7fa590100af0 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa58e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 0x7fa59019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa58e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 0x7fa59019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33258/0 (socket says 172.21.15.37:33258) 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa58ed9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 0x7fa59019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa58e59c640 1 -- 172.21.15.37:0/2736453946 learned_addr learned my addr 172.21.15.37:0/2736453946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.659+0000 7fa58f59e640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa590198610 0x7fa590198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58e59c640 1 -- 172.21.15.37:0/2736453946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 msgr2=0x7fa59019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58e59c640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 0x7fa59019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58e59c640 1 -- 172.21.15.37:0/2736453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa590198610 msgr2=0x7fa590198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58e59c640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa590198610 0x7fa590198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58e59c640 1 -- 172.21.15.37:0/2736453946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa590199130 con 0x7fa5901072b0 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58ed9d640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 0x7fa59019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa58e59c640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 0x7fa59019e950 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa584032fd0 tx=0x7fa584031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58403f030 con 0x7fa5901072b0 2024-04-23T18:34:52.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5901993c0 con 0x7fa5901072b0 2024-04-23T18:34:52.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa584004490 con 0x7fa5901072b0 2024-04-23T18:34:52.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584035070 con 0x7fa5901072b0 2024-04-23T18:34:52.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.660+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5901a5560 con 0x7fa5901072b0 2024-04-23T18:34:52.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.661+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5900691c0 con 0x7fa5901072b0 2024-04-23T18:34:52.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.662+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa584042030 con 0x7fa5901072b0 2024-04-23T18:34:52.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.663+0000 7fa577fff640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa5780a3d20 0x7fa5780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:52.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.663+0000 7fa58ed9d640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa5780a3d20 0x7fa5780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:52.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.663+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8793+0+0 (secure 0 0 0) 0x7fa5840f0c90 con 0x7fa5901072b0 2024-04-23T18:34:52.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.663+0000 7fa58ed9d640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa5780a3d20 0x7fa5780a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa580009fc0 tx=0x7fa580005f90 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:52.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.668+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5840bcb70 con 0x7fa5901072b0 2024-04-23T18:34:52.837 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: ignoring --setuser ceph since I am not root 2024-04-23T18:34:52.837 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: ignoring --setgroup ceph since I am not root 2024-04-23T18:34:52.837 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mds[62214]: -- 172.21.15.172:0/1063004245 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556671abba40 con 0x556671a8ec00 2024-04-23T18:34:52.837 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: starting mds.b at 2024-04-23T18:34:52.838 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e[61583]: ignoring --setuser ceph since I am not root 2024-04-23T18:34:52.838 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e[61583]: ignoring --setgroup ceph since I am not root 2024-04-23T18:34:52.838 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mds[61587]: -- 172.21.15.172:0/2467383355 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596b901da40 con 0x5596b8ff1000 2024-04-23T18:34:52.838 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e[61583]: starting mds.e at 2024-04-23T18:34:52.838 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h[61890]: ignoring --setuser ceph since I am not root 2024-04-23T18:34:52.838 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h[61890]: ignoring --setgroup ceph since I am not root 2024-04-23T18:34:52.839 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-mds[61894]: -- 172.21.15.172:0/68384838 <== mon.2 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559dfd679a40 con 0x559dfd64d000 2024-04-23T18:34:52.839 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:34:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h[61890]: starting mds.h at 2024-04-23T18:34:52.854 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: ignoring --setuser ceph since I am not root 2024-04-23T18:34:52.854 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: ignoring --setgroup ceph since I am not root 2024-04-23T18:34:52.854 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mds[63204]: -- 172.21.15.173:0/3148511953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55562b8cba40 con 0x55562b89e800 2024-04-23T18:34:52.854 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: starting mds.c at 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i[63509]: ignoring --setuser ceph since I am not root 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i[63509]: ignoring --setgroup ceph since I am not root 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mds[63513]: -- 172.21.15.173:0/2106696476 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608297cda40 con 0x5608297a0800 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i[63509]: starting mds.i at 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f[62886]: ignoring --setuser ceph since I am not root 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f[62886]: ignoring --setgroup ceph since I am not root 2024-04-23T18:34:52.855 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-mds[62890]: -- 172.21.15.173:0/885511222 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555751bdba40 con 0x555751baec00 2024-04-23T18:34:52.856 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:34:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f[62886]: starting mds.f at 2024-04-23T18:34:52.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a[29400]: 2024-04-23T18:34:52.534+0000 7f5ba209b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:34:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:52.895+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa59010a490 con 0x7fa5901072b0 2024-04-23T18:34:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.543+0000 7fa577fff640 1 -- 172.21.15.37:0/2736453946 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7fa5840c0e70 con 0x7fa5901072b0 2024-04-23T18:34:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa5780a3d20 msgr2=0x7fa5780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa5780a3d20 0x7fa5780a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa580009fc0 tx=0x7fa580005f90 comp rx=0 tx=0).stop 2024-04-23T18:34:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 msgr2=0x7fa59019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 0x7fa59019e950 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa584032fd0 tx=0x7fa584031a20 comp rx=0 tx=0).stop 2024-04-23T18:34:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 shutdown_connections 2024-04-23T18:34:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7fa5780a3d20 0x7fa5780a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa590198610 0x7fa590198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5901072b0 0x7fa59019e950 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 --2- 172.21.15.37:0/2736453946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa590100af0 0x7fa59019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 >> 172.21.15.37:0/2736453946 conn(0x7fa5900760e0 msgr2=0x7fa5900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 shutdown_connections 2024-04-23T18:34:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.546+0000 7fa5951f7640 1 -- 172.21.15.37:0/2736453946 wait complete. 2024-04-23T18:34:53.551 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1331194653' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:34:53.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:34:53.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:34:53.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1331194653' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:34:53.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:34:53.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-23T18:34:53.552 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:53 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2736453946' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:34:53.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T18:34:53.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1331194653' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:34:53.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:34:53.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:34:53.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1331194653' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:34:53.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:34:53.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-23T18:34:53.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:53 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2736453946' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1331194653' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1331194653' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-23T18:34:53.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:53 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2736453946' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:34:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 -- 172.21.15.37:0/4129528747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06af10 msgr2=0x7f005c10edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 --2- 172.21.15.37:0/4129528747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06af10 0x7f005c10edf0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0054009cd0 tx=0x7f005402a970 comp rx=0 tx=0).stop 2024-04-23T18:34:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 -- 172.21.15.37:0/4129528747 shutdown_connections 2024-04-23T18:34:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 --2- 172.21.15.37:0/4129528747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06af10 0x7f005c10edf0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 --2- 172.21.15.37:0/4129528747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c06a550 0x7f005c06a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 --2- 172.21.15.37:0/4129528747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 0x7f005c108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 -- 172.21.15.37:0/4129528747 >> 172.21.15.37:0/4129528747 conn(0x7f005c076c50 msgr2=0x7f005c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 -- 172.21.15.37:0/4129528747 shutdown_connections 2024-04-23T18:34:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.674+0000 7f0061235640 1 -- 172.21.15.37:0/4129528747 wait complete. 2024-04-23T18:34:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 Processor -- start 2024-04-23T18:34:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 -- start start 2024-04-23T18:34:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 0x7f005c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c06af10 0x7f005c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 0x7f005c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1121b0 con 0x7f005c06a550 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f005c112030 con 0x7f005c108150 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f0061235640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f005c112330 con 0x7f005c06af10 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.675+0000 7f005ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 0x7f005c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 0x7f005c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33268/0 (socket says 172.21.15.37:33268) 2024-04-23T18:34:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 -- 172.21.15.37:0/1339935364 learned_addr learned my addr 172.21.15.37:0/1339935364 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005a59c640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c06af10 0x7f005c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005b59e640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 0x7f005c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 -- 172.21.15.37:0/1339935364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c06af10 msgr2=0x7f005c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c06af10 0x7f005c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 -- 172.21.15.37:0/1339935364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 msgr2=0x7f005c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 0x7f005c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 -- 172.21.15.37:0/1339935364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c19d6b0 con 0x7f005c06a550 2024-04-23T18:34:53.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005b59e640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 0x7f005c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f005ad9d640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 0x7f005c198710 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f004c00d770 tx=0x7f004c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c017070 con 0x7f005c06a550 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c19d9a0 con 0x7f005c06a550 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004c009c30 con 0x7f005c06a550 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c003cc0 con 0x7f005c06a550 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.676+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1a5230 con 0x7f005c06a550 2024-04-23T18:34:53.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.677+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c103680 con 0x7f005c06a550 2024-04-23T18:34:53.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.679+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f004c003e60 con 0x7f005c06a550 2024-04-23T18:34:53.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.679+0000 7f0043fff640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f00440a3d20 0x7f00440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:53.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.679+0000 7f005a59c640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f00440a3d20 0x7f00440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:53.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.679+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8793+0+0 (secure 0 0 0) 0x7f004c0d06c0 con 0x7f005c06a550 2024-04-23T18:34:53.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.680+0000 7f005a59c640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f00440a3d20 0x7f00440a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f005c107690 tx=0x7f005000b390 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.683+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f004c009de0 con 0x7f005c06a550 2024-04-23T18:34:53.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:53.869+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f005c199a10 con 0x7f005c06a550 2024-04-23T18:34:54.505 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: pgmap v454: 97 pgs: 97 active+clean; 763 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 2.6 MiB/s wr, 78 op/s 2024-04-23T18:34:54.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:34:54.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:34:54.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: Cluster is now healthy 2024-04-23T18:34:54.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/2736453946' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:34:54.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6834/2941665564,v1:172.21.15.173:6835/2941665564] up:boot 2024-04-23T18:34:54.506 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6834/35676625,v1:172.21.15.172:6835/35676625] up:boot 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6836/558101858,v1:172.21.15.172:6837/558101858] up:boot 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.172:6838/3006058597,v1:172.21.15.172:6839/3006058597] up:boot 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6837/2107731904,v1:172.21.15.173:6839/2107731904] up:boot 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: mds.? [v2:172.21.15.173:6836/872264261,v1:172.21.15.173:6838/872264261] up:boot 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: fsmap 12 up:standby 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:34:54.507 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:34:54.508 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:34:54.508 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:34:54.508 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:54 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1339935364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:34:54.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.545+0000 7f0043fff640 1 -- 172.21.15.37:0/1339935364 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f004c09cc50 con 0x7f005c06a550 2024-04-23T18:34:54.538 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-23T18:34:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.549+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f00440a3d20 msgr2=0x7f00440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.549+0000 7f0061235640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f00440a3d20 0x7f00440a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f005c107690 tx=0x7f005000b390 comp rx=0 tx=0).stop 2024-04-23T18:34:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.549+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 msgr2=0x7f005c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.549+0000 7f0061235640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 0x7f005c198710 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f004c00d770 tx=0x7f004c00dc30 comp rx=0 tx=0).stop 2024-04-23T18:34:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.549+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 shutdown_connections 2024-04-23T18:34:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f00440a3d20 0x7f00440a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f005c108150 0x7f005c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f005c06af10 0x7f005c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 --2- 172.21.15.37:0/1339935364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f005c06a550 0x7f005c198710 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 >> 172.21.15.37:0/1339935364 conn(0x7f005c076c50 msgr2=0x7f005c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 shutdown_connections 2024-04-23T18:34:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.550+0000 7f0061235640 1 -- 172.21.15.37:0/1339935364 wait complete. 2024-04-23T18:34:54.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: pgmap v454: 97 pgs: 97 active+clean; 763 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 2.6 MiB/s wr, 78 op/s 2024-04-23T18:34:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:34:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:34:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: Cluster is now healthy 2024-04-23T18:34:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/2736453946' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:34:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6834/2941665564,v1:172.21.15.173:6835/2941665564] up:boot 2024-04-23T18:34:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6834/35676625,v1:172.21.15.172:6835/35676625] up:boot 2024-04-23T18:34:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6836/558101858,v1:172.21.15.172:6837/558101858] up:boot 2024-04-23T18:34:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.172:6838/3006058597,v1:172.21.15.172:6839/3006058597] up:boot 2024-04-23T18:34:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6837/2107731904,v1:172.21.15.173:6839/2107731904] up:boot 2024-04-23T18:34:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: mds.? [v2:172.21.15.173:6836/872264261,v1:172.21.15.173:6838/872264261] up:boot 2024-04-23T18:34:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: fsmap 12 up:standby 2024-04-23T18:34:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:34:54.564 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:34:54.564 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:34:54.564 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:34:54.564 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:34:54.564 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:34:54.565 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:54 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1339935364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:34:54.566 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T18:34:54.586 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: pgmap v454: 97 pgs: 97 active+clean; 763 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 2.6 MiB/s wr, 78 op/s 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: Cluster is now healthy 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/2736453946' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6834/2941665564,v1:172.21.15.173:6835/2941665564] up:boot 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6834/35676625,v1:172.21.15.172:6835/35676625] up:boot 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6836/558101858,v1:172.21.15.172:6837/558101858] up:boot 2024-04-23T18:34:54.587 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.172:6838/3006058597,v1:172.21.15.172:6839/3006058597] up:boot 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6837/2107731904,v1:172.21.15.173:6839/2107731904] up:boot 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: mds.? [v2:172.21.15.173:6836/872264261,v1:172.21.15.173:6838/872264261] up:boot 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: fsmap 12 up:standby 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:34:54.588 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='mgr.15270 172.21.15.37:0/1553918918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:34:54.589 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:54 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1339935364' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:34:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 -- 172.21.15.37:0/18174172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9348103af0 msgr2=0x7f934810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 --2- 172.21.15.37:0/18174172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9348103af0 0x7f934810bfd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9338009900 tx=0x7f933802a510 comp rx=0 tx=0).stop 2024-04-23T18:34:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 -- 172.21.15.37:0/18174172 shutdown_connections 2024-04-23T18:34:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 --2- 172.21.15.37:0/18174172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f934810c510 0x7f934810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 --2- 172.21.15.37:0/18174172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9348103af0 0x7f934810bfd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 --2- 172.21.15.37:0/18174172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93481031d0 0x7f93481035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 -- 172.21.15.37:0/18174172 >> 172.21.15.37:0/18174172 conn(0x7f9348077f50 msgr2=0x7f9348078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 -- 172.21.15.37:0/18174172 shutdown_connections 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 -- 172.21.15.37:0/18174172 wait complete. 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 Processor -- start 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.690+0000 7f934fe4e640 1 -- start start 2024-04-23T18:34:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934fe4e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93481031d0 0x7f9348198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934fe4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9348103af0 0x7f9348198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934fe4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 0x7f934819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934fe4e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93481121a0 con 0x7f934810c510 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934fe4e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9348112020 con 0x7f9348103af0 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934fe4e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9348112320 con 0x7f93481031d0 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 0x7f934819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934dbea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93481031d0 0x7f9348198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 0x7f934819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41890/0 (socket says 172.21.15.37:41890) 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 -- 172.21.15.37:0/1599569112 learned_addr learned my addr 172.21.15.37:0/1599569112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934d3e9640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9348103af0 0x7f9348198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 -- 172.21.15.37:0/1599569112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93481031d0 msgr2=0x7f9348198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93481031d0 0x7f9348198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 -- 172.21.15.37:0/1599569112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9348103af0 msgr2=0x7f9348198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9348103af0 0x7f9348198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 -- 172.21.15.37:0/1599569112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934819d710 con 0x7f934810c510 2024-04-23T18:34:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934d3e9640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9348103af0 0x7f9348198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f934e3eb640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 0x7f934819cfe0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f934000eb10 tx=0x7f934000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.691+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340004b20 con 0x7f934810c510 2024-04-23T18:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.692+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9340004cc0 con 0x7f934810c510 2024-04-23T18:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.692+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934819da00 con 0x7f934810c510 2024-04-23T18:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.692+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340010670 con 0x7f934810c510 2024-04-23T18:34:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.692+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9348100d50 con 0x7f934810c510 2024-04-23T18:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.693+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9348068d70 con 0x7f934810c510 2024-04-23T18:34:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.694+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9340005020 con 0x7f934810c510 2024-04-23T18:34:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.695+0000 7f933effd640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93340a3d20 0x7f93340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.695+0000 7f934dbea640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93340a3d20 0x7f93340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.695+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8239+0+0 (secure 0 0 0) 0x7f93400d0bc0 con 0x7f934810c510 2024-04-23T18:34:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.695+0000 7f934dbea640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93340a3d20 0x7f93340a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9344010d80 tx=0x7f9344012040 comp rx=0 tx=0).ready entity=mgr.15270 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.699+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f934009d380 con 0x7f934810c510 2024-04-23T18:34:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:54.890+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9348107690 con 0x7f934810c510 2024-04-23T18:34:55.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.547+0000 7f933effd640 1 -- 172.21.15.37:0/1599569112 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7f934009cc80 con 0x7f934810c510 2024-04-23T18:34:55.539 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-23T18:34:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93340a3d20 msgr2=0x7f93340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93340a3d20 0x7f93340a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9344010d80 tx=0x7f9344012040 comp rx=0 tx=0).stop 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 msgr2=0x7f934819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 0x7f934819cfe0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f934000eb10 tx=0x7f934000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 shutdown_connections 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:6800/816900776,v1:172.21.15.37:6801/816900776] conn(0x7f93340a3d20 0x7f93340a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f934810c510 0x7f934819cfe0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9348103af0 0x7f9348198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 --2- 172.21.15.37:0/1599569112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93481031d0 0x7f9348198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.549+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 >> 172.21.15.37:0/1599569112 conn(0x7f9348077f50 msgr2=0x7f93480fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.550+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 shutdown_connections 2024-04-23T18:34:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-23T18:34:55.550+0000 7f934fe4e640 1 -- 172.21.15.37:0/1599569112 wait complete. 2024-04-23T18:34:55.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:34:55.578 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:34:55.604 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:34:55.630 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T18:34:55.631 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:34:55.645 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:34:55.660 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:34:55.686 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T18:34:55.686 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-23T18:34:55.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:34:55.797 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:55 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1339935364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:34:55.797 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:55 smithi173 ceph-mon[32906]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:34:55.797 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:55 smithi173 ceph-mon[32906]: pgmap v456: 65 pgs: 65 active+clean; 253 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:55.798 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:55 smithi173 ceph-mon[32906]: from='client.? 172.21.15.37:0/1599569112' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:34:55.836 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:55 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1339935364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:34:55.836 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:55 smithi172 ceph-mon[32456]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:34:55.836 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:55 smithi172 ceph-mon[32456]: pgmap v456: 65 pgs: 65 active+clean; 253 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:55.836 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:55 smithi172 ceph-mon[32456]: from='client.? 172.21.15.37:0/1599569112' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:34:55.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1339935364' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T18:34:55.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 ceph-mon[29404]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:34:55.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 ceph-mon[29404]: pgmap v456: 65 pgs: 65 active+clean; 253 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:34:55.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 ceph-mon[29404]: from='client.? 172.21.15.37:0/1599569112' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T18:34:55.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 systemd[1]: Stopping Ceph mon.a for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:34:56.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a[29400]: 2024-04-23T18:34:55.995+0000 7f5ba78a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:34:56.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a[29400]: 2024-04-23T18:34:55.995+0000 7f5ba78a6700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T18:34:56.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:56 smithi037 podman[90625]: 2024-04-23 18:34:56.026755422 +0000 UTC m=+0.143526964 container died 21630b3f612def12b1562f49f04d91e1cce3b7e55f632732aa1469a14313c397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:34:56.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:56 smithi037 podman[90625]: 2024-04-23 18:34:56.387634697 +0000 UTC m=+0.504406228 container cleanup 21630b3f612def12b1562f49f04d91e1cce3b7e55f632732aa1469a14313c397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-04-23T18:34:56.620 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 23 18:34:56 smithi037 bash[90625]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:34:56.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.a.service' 2024-04-23T18:34:56.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:34:56.870 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-23T18:34:56.870 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-23T18:34:56.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.b 2024-04-23T18:34:57.188 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:56 smithi172 systemd[1]: Stopping Ceph mon.b for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:34:57.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-b[32452]: 2024-04-23T18:34:57.195+0000 7f949251e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:34:57.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-b[32452]: 2024-04-23T18:34:57.195+0000 7f949251e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-23T18:34:57.453 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:57 smithi172 podman[68777]: 2024-04-23 18:34:57.257142942 +0000 UTC m=+0.182989081 container died 3ae5bb191b5d4b76321452d92fabf0af8736008cab107bc08736c9742c272ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:34:57.731 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:57 smithi172 podman[68777]: 2024-04-23 18:34:57.607115728 +0000 UTC m=+0.532961874 container cleanup 3ae5bb191b5d4b76321452d92fabf0af8736008cab107bc08736c9742c272ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T18:34:57.732 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:57 smithi172 bash[68777]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-b 2024-04-23T18:34:58.030 INFO:journalctl@ceph.mon.b.smithi172.stdout:Apr 23 18:34:57 smithi172 podman[68790]: 2024-04-23 18:34:57.916932889 +0000 UTC m=+0.661165378 container remove 3ae5bb191b5d4b76321452d92fabf0af8736008cab107bc08736c9742c272ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:34:58.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.b.service' 2024-04-23T18:34:58.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:34:58.097 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-23T18:34:58.097 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-23T18:34:58.098 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.c 2024-04-23T18:34:58.405 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:58 smithi173 systemd[1]: Stopping Ceph mon.c for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:34:58.854 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c[32901]: 2024-04-23T18:34:58.412+0000 7f3b0edaa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:34:58.854 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c[32901]: 2024-04-23T18:34:58.412+0000 7f3b0edaa700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-23T18:34:58.854 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:58 smithi173 podman[70523]: 2024-04-23 18:34:58.617909193 +0000 UTC m=+0.325340016 container died ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T18:34:59.349 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:59 smithi173 podman[70523]: 2024-04-23 18:34:59.164908031 +0000 UTC m=+0.872338901 container cleanup ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-23T18:34:59.350 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:59 smithi173 bash[70523]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c 2024-04-23T18:34:59.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.c.service' 2024-04-23T18:34:59.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:59 smithi173 podman[70537]: 2024-04-23 18:34:59.358220935 +0000 UTC m=+0.736466955 container remove ce753cebb77b2b84ef48867b5cb97578c28e7d95148f43a0ca693750a7a4b527 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:34:59.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:59 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-23T18:34:59.603 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:59 smithi173 systemd[1]: Stopped Ceph mon.c for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:34:59.604 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 23 18:34:59 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mon.c.service: Consumed 15.721s CPU time. 2024-04-23T18:34:59.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:34:59.624 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-23T18:34:59.624 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-23T18:34:59.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:34:59.917 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:34:59 smithi037 systemd[1]: Stopping Ceph mgr.x for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:00.217 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:34:59 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x[29632]: 2024-04-23T18:34:59.923+0000 7fbbae811700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:00.217 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:34:59 smithi037 podman[90745]: 2024-04-23 18:34:59.956464091 +0000 UTC m=+0.128546781 container died 6e1ef653973db5c81db09e202dc794d746aeb25a499e76887add9131b81c3cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T18:35:00.476 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:35:00 smithi037 podman[90745]: 2024-04-23 18:35:00.300840484 +0000 UTC m=+0.472923166 container cleanup 6e1ef653973db5c81db09e202dc794d746aeb25a499e76887add9131b81c3cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-23T18:35:00.477 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:35:00 smithi037 bash[90745]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x 2024-04-23T18:35:00.718 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-23T18:35:00.739 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:35:00 smithi037 podman[90759]: 2024-04-23 18:35:00.485579341 +0000 UTC m=+0.526430829 container remove 6e1ef653973db5c81db09e202dc794d746aeb25a499e76887add9131b81c3cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T18:35:00.739 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:35:00 smithi037 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-23T18:35:00.739 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:35:00 smithi037 systemd[1]: Stopped Ceph mgr.x for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:35:00.739 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 23 18:35:00 smithi037 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.x.service: Consumed 1min 31.291s CPU time. 2024-04-23T18:35:00.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:00.753 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-23T18:35:00.753 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-23T18:35:00.753 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.y 2024-04-23T18:35:01.082 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:35:00 smithi172 systemd[1]: Stopping Ceph mgr.y for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:01.336 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:35:01 smithi172 podman[68909]: 2024-04-23 18:35:01.090629206 +0000 UTC m=+0.136304560 container died 5716a8fb5a754b9e56185470fd6a89d17213d271ea895c2577a0a28693ea6d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:35:01.704 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:35:01 smithi172 podman[68909]: 2024-04-23 18:35:01.46178827 +0000 UTC m=+0.507463623 container cleanup 5716a8fb5a754b9e56185470fd6a89d17213d271ea895c2577a0a28693ea6d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T18:35:01.704 INFO:journalctl@ceph.mgr.y.smithi172.stdout:Apr 23 18:35:01 smithi172 bash[68909]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-y 2024-04-23T18:35:01.871 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-23T18:35:01.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:01.906 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-23T18:35:01.906 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-23T18:35:01.906 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.z 2024-04-23T18:35:02.232 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:35:01 smithi173 systemd[1]: Stopping Ceph mgr.z for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:02.520 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:35:02 smithi173 podman[70645]: 2024-04-23 18:35:02.240427743 +0000 UTC m=+0.131843228 container died 3348114d907dc1e104a0fe47c2827f3faf905cf48b42a680e3a24cdfecac955d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:35:02.854 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:35:02 smithi173 podman[70645]: 2024-04-23 18:35:02.628157857 +0000 UTC m=+0.519573341 container cleanup 3348114d907dc1e104a0fe47c2827f3faf905cf48b42a680e3a24cdfecac955d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:02.854 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 23 18:35:02 smithi173 bash[70645]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mgr-z 2024-04-23T18:35:03.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-23T18:35:03.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:03.092 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-23T18:35:03.093 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T18:35:03.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.0 2024-04-23T18:35:03.422 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:03 smithi037 systemd[1]: Stopping Ceph osd.0 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:03.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0[41855]: 2024-04-23T18:35:03.429+0000 7fe8d9710700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:03.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0[41855]: 2024-04-23T18:35:03.429+0000 7fe8d9710700 -1 osd.0 113 *** Got signal Terminated *** 2024-04-23T18:35:03.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0[41855]: 2024-04-23T18:35:03.429+0000 7fe8d9710700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:09.276 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:08 smithi037 podman[90865]: 2024-04-23 18:35:08.90473587 +0000 UTC m=+5.593068832 container died 107101d733063e72824f8ded486cc5c09c2aac82563f0950586005b0e25b8e89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:35:09.615 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:09 smithi037 podman[90865]: 2024-04-23 18:35:09.370430557 +0000 UTC m=+6.058763525 container cleanup 107101d733063e72824f8ded486cc5c09c2aac82563f0950586005b0e25b8e89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:09.615 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:09 smithi037 bash[90865]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0 2024-04-23T18:35:09.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:09 smithi037 podman[90882]: 2024-04-23 18:35:09.623745676 +0000 UTC m=+0.715971781 container remove 107101d733063e72824f8ded486cc5c09c2aac82563f0950586005b0e25b8e89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T18:35:10.202 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:09 smithi037 podman[90973]: 2024-04-23 18:35:09.85672389 +0000 UTC m=+0.018577441 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:10.202 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:09 smithi037 podman[90973]: 2024-04-23 18:35:09.96635628 +0000 UTC m=+0.128209830 container create 56fad0ae597ab69e970247fb8800877ddc97d4d499645d25441769a313000285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:10.202 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:10 smithi037 podman[90973]: 2024-04-23 18:35:10.210487076 +0000 UTC m=+0.372340627 container init 56fad0ae597ab69e970247fb8800877ddc97d4d499645d25441769a313000285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:35:10.515 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:10 smithi037 podman[90973]: 2024-04-23 18:35:10.214019789 +0000 UTC m=+0.375873342 container start 56fad0ae597ab69e970247fb8800877ddc97d4d499645d25441769a313000285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:35:10.515 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:10 smithi037 podman[90973]: 2024-04-23 18:35:10.285473536 +0000 UTC m=+0.447327103 container attach 56fad0ae597ab69e970247fb8800877ddc97d4d499645d25441769a313000285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T18:35:10.796 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 23 18:35:10 smithi037 podman[90973]: 2024-04-23 18:35:10.524419608 +0000 UTC m=+0.686273160 container died 56fad0ae597ab69e970247fb8800877ddc97d4d499645d25441769a313000285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:11.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T18:35:11.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:11.134 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T18:35:11.134 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T18:35:11.135 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.1 2024-04-23T18:35:11.466 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:11 smithi172 systemd[1]: Stopping Ceph osd.1 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:11.836 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1[37033]: 2024-04-23T18:35:11.473+0000 7f9442117700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:11.836 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1[37033]: 2024-04-23T18:35:11.473+0000 7f9442117700 -1 osd.1 113 *** Got signal Terminated *** 2024-04-23T18:35:11.836 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1[37033]: 2024-04-23T18:35:11.473+0000 7f9442117700 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:17.189 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:16 smithi172 podman[69031]: 2024-04-23 18:35:16.87922911 +0000 UTC m=+5.540917775 container died 25e7b8fcd36492599df23970fdb08a91ebd7a2fbbd657492f20686332afa80b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:35:17.586 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:17 smithi172 podman[69031]: 2024-04-23 18:35:17.198066359 +0000 UTC m=+5.859755022 container cleanup 25e7b8fcd36492599df23970fdb08a91ebd7a2fbbd657492f20686332afa80b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-23T18:35:17.586 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:17 smithi172 bash[69031]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1 2024-04-23T18:35:17.586 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:17 smithi172 podman[69049]: 2024-04-23 18:35:17.407835695 +0000 UTC m=+0.526077485 container remove 25e7b8fcd36492599df23970fdb08a91ebd7a2fbbd657492f20686332afa80b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:35:18.086 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:17 smithi172 podman[69142]: 2024-04-23 18:35:17.727412408 +0000 UTC m=+0.018082937 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:18.086 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:17 smithi172 podman[69142]: 2024-04-23 18:35:17.825216402 +0000 UTC m=+0.115886970 container create 8ab3f8e7e5f211bfe7f1cd0e784598df59142b068c56047636f5cf7026c90cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:35:18.087 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 podman[69142]: 2024-04-23 18:35:18.035756208 +0000 UTC m=+0.326426731 container init 8ab3f8e7e5f211bfe7f1cd0e784598df59142b068c56047636f5cf7026c90cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-23T18:35:18.087 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 podman[69142]: 2024-04-23 18:35:18.038733839 +0000 UTC m=+0.329404373 container start 8ab3f8e7e5f211bfe7f1cd0e784598df59142b068c56047636f5cf7026c90cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:35:18.586 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 podman[69142]: 2024-04-23 18:35:18.129570768 +0000 UTC m=+0.420241321 container attach 8ab3f8e7e5f211bfe7f1cd0e784598df59142b068c56047636f5cf7026c90cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:35:18.586 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 podman[69142]: 2024-04-23 18:35:18.33605353 +0000 UTC m=+0.626724054 container died 8ab3f8e7e5f211bfe7f1cd0e784598df59142b068c56047636f5cf7026c90cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T18:35:18.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T18:35:18.881 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 podman[69168]: 2024-04-23 18:35:18.847874915 +0000 UTC m=+0.504784249 container remove 8ab3f8e7e5f211bfe7f1cd0e784598df59142b068c56047636f5cf7026c90cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T18:35:18.881 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-23T18:35:18.881 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 systemd[1]: Stopped Ceph osd.1 for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:35:18.881 INFO:journalctl@ceph.osd.1.smithi172.stdout:Apr 23 18:35:18 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.1.service: Consumed 49.845s CPU time. 2024-04-23T18:35:18.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:18.896 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T18:35:18.896 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T18:35:18.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.2 2024-04-23T18:35:19.183 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:18 smithi173 systemd[1]: Stopping Ceph osd.2 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:19.603 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2[38068]: 2024-04-23T18:35:19.191+0000 7fe1c3348700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:19.604 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2[38068]: 2024-04-23T18:35:19.191+0000 7fe1c3348700 -1 osd.2 113 *** Got signal Terminated *** 2024-04-23T18:35:19.604 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2[38068]: 2024-04-23T18:35:19.191+0000 7fe1c3348700 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:24.958 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:24 smithi173 podman[70769]: 2024-04-23 18:35:24.665095247 +0000 UTC m=+5.565158849 container died ebbff91f9e63e58eff43f97819e593db0e5578c35a1725e0ccc380f68e90038a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:25.354 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:24 smithi173 podman[70769]: 2024-04-23 18:35:24.966577088 +0000 UTC m=+5.866640690 container cleanup ebbff91f9e63e58eff43f97819e593db0e5578c35a1725e0ccc380f68e90038a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-23T18:35:25.354 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:24 smithi173 bash[70769]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2 2024-04-23T18:35:25.354 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:25 smithi173 podman[70786]: 2024-04-23 18:35:25.159871871 +0000 UTC m=+0.497878236 container remove ebbff91f9e63e58eff43f97819e593db0e5578c35a1725e0ccc380f68e90038a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:35:25.854 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:25 smithi173 podman[70876]: 2024-04-23 18:35:25.485674854 +0000 UTC m=+0.017778391 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:25.854 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:25 smithi173 podman[70876]: 2024-04-23 18:35:25.594274075 +0000 UTC m=+0.126377597 container create 3afa4f0e6c521fb5eb99a14ef33ad671bff98b64f30219508d868fe7401724fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:25.854 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:25 smithi173 podman[70876]: 2024-04-23 18:35:25.822450526 +0000 UTC m=+0.354554059 container init 3afa4f0e6c521fb5eb99a14ef33ad671bff98b64f30219508d868fe7401724fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T18:35:25.854 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:25 smithi173 podman[70876]: 2024-04-23 18:35:25.825859217 +0000 UTC m=+0.357962749 container start 3afa4f0e6c521fb5eb99a14ef33ad671bff98b64f30219508d868fe7401724fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:35:26.115 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:25 smithi173 podman[70876]: 2024-04-23 18:35:25.906239331 +0000 UTC m=+0.438342867 container attach 3afa4f0e6c521fb5eb99a14ef33ad671bff98b64f30219508d868fe7401724fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:26.392 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 23 18:35:26 smithi173 podman[70876]: 2024-04-23 18:35:26.124715066 +0000 UTC m=+0.656818590 container died 3afa4f0e6c521fb5eb99a14ef33ad671bff98b64f30219508d868fe7401724fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:35:26.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T18:35:26.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:26.706 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T18:35:26.707 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T18:35:26.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.3 2024-04-23T18:35:27.008 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:26 smithi037 systemd[1]: Stopping Ceph osd.3 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:27.369 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:27 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:35:27.016+0000 7f88b0cc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:27.369 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:27 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:35:27.016+0000 7f88b0cc4700 -1 osd.3 113 *** Got signal Terminated *** 2024-04-23T18:35:27.369 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:27 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:35:27.016+0000 7f88b0cc4700 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:29.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:29.285+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:30.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:30.036+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:30.369 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:35:30.328+0000 7f88ad2b8700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:04.618385+0000 front 2024-04-23T18:35:04.618355+0000 (oldest deadline 2024-04-23T18:35:29.918539+0000) 2024-04-23T18:35:30.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:30 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:30.111+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:30.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:30.255+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:31.317 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:31 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:31.068+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:31.353 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:31.046+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:31.584 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:31 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3[48525]: 2024-04-23T18:35:31.325+0000 7f88ad2b8700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:04.618385+0000 front 2024-04-23T18:35:04.618355+0000 (oldest deadline 2024-04-23T18:35:29.918539+0000) 2024-04-23T18:35:31.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:31.296+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:31.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:31.426+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:31.870 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:31 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:31.592+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:32.353 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:32.035+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:32.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:32 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:32.107+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:32.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:32.257+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:32.767 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:32 smithi037 podman[91093]: 2024-04-23 18:35:32.522741056 +0000 UTC m=+5.600939383 container died 509dbb5b676da13a192fa7d8c0194a3e08080957f95431a373cbd4b2fd35957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:32.767 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:32 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:32.640+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:32.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:32.404+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:33.075 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:32 smithi037 podman[91093]: 2024-04-23 18:35:32.894094727 +0000 UTC m=+5.972293051 container cleanup 509dbb5b676da13a192fa7d8c0194a3e08080957f95431a373cbd4b2fd35957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-23T18:35:33.076 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:32 smithi037 bash[91093]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3 2024-04-23T18:35:33.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:33.053+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:33.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:33 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:33.083+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:33.369 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:33 smithi037 podman[91111]: 2024-04-23 18:35:33.184914375 +0000 UTC m=+0.662743505 container remove 509dbb5b676da13a192fa7d8c0194a3e08080957f95431a373cbd4b2fd35957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:33.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:33 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:33.246+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:33.770 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:33 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:33.672+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:33.771 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:33 smithi037 podman[91204]: 2024-04-23 18:35:33.429670081 +0000 UTC m=+0.018583497 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:33.771 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:33 smithi037 podman[91204]: 2024-04-23 18:35:33.543934888 +0000 UTC m=+0.132848286 container create 0e2f9b9ce2a1fd869baf39aabb4bf4155f559583f9cafcce2cd10b311e31b7e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:33.771 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:33 smithi037 podman[91204]: 2024-04-23 18:35:33.779449208 +0000 UTC m=+0.368362608 container init 0e2f9b9ce2a1fd869baf39aabb4bf4155f559583f9cafcce2cd10b311e31b7e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:35:33.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:33.429+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:34.078 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:33 smithi037 podman[91204]: 2024-04-23 18:35:33.783268378 +0000 UTC m=+0.372181770 container start 0e2f9b9ce2a1fd869baf39aabb4bf4155f559583f9cafcce2cd10b311e31b7e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-23T18:35:34.078 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:33 smithi037 podman[91204]: 2024-04-23 18:35:33.871340489 +0000 UTC m=+0.460253903 container attach 0e2f9b9ce2a1fd869baf39aabb4bf4155f559583f9cafcce2cd10b311e31b7e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:35:34.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:34.070+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:34.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:34 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:34.085+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:34.370 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 23 18:35:34 smithi037 podman[91204]: 2024-04-23 18:35:34.091139246 +0000 UTC m=+0.680052645 container died 0e2f9b9ce2a1fd869baf39aabb4bf4155f559583f9cafcce2cd10b311e31b7e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T18:35:34.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:34 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:34.270+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:34.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T18:35:34.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:34.711 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T18:35:34.711 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T18:35:34.711 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.4 2024-04-23T18:35:34.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:34.423+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:35.008 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:34 smithi172 systemd[1]: Stopping Ceph osd.4 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:35.046 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:34 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:34.696+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:35.307 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:35 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4[43831]: 2024-04-23T18:35:35.016+0000 7fe8237b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:35.308 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:35 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4[43831]: 2024-04-23T18:35:35.016+0000 7fe8237b8700 -1 osd.4 113 *** Got signal Terminated *** 2024-04-23T18:35:35.308 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:35 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4[43831]: 2024-04-23T18:35:35.016+0000 7fe8237b8700 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:35.339 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:35.054+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:35.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:35.030+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:35.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:35 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:35.315+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:35.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:35.436+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:36.076 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:35 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:35.705+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:36.353 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:36.048+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:36.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:36 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:36.085+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:36.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:36 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:36.320+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:36.834 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:36.430+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:37.104 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:37.053+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:37.118 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:36 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:36.663+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:37.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:37 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:37.127+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:37.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:37 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:37.338+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:37.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:37 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:37.548+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:37.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:37.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:38.119 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:37 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:37.669+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:38.353 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:38.031+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:38.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:38 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:38.537+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:38.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:38 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:38.330+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:38.619 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:38.130+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:38.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:38.366+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:38.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:38.366+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:39.089 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:38.709+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:39.089 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:38 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:38.709+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:39.319 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:39.066+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:39.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:39.097+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:39.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:39.097+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:39.556 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:39 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:39.306+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:39.556 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:39 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:39.306+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:39.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:39.327+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:39.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:39.327+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:39.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:39.477+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:39.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:39 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:39.564+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:40.066 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:39.704+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:40.066 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:39 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:39.704+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:40.304 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:40.047+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:40.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:40.074+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:40.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:40.074+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:40.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:40.430+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:40.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:40.312+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:40.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:40.312+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:40.784 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:40 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:40.549+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:40.785 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:40 smithi172 podman[69260]: 2024-04-23 18:35:40.498963554 +0000 UTC m=+5.583423258 container died 2b34095d97f5541e7492995cd6fff4dbddef837afa954db59e9c72bfd4c30d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T18:35:40.785 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:40 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:40.348+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:40.785 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:40 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:40.348+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:41.048 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:40.685+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:41.048 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:40 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:40.685+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:41.086 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:40 smithi172 podman[69260]: 2024-04-23 18:35:40.866643518 +0000 UTC m=+5.951103221 container cleanup 2b34095d97f5541e7492995cd6fff4dbddef837afa954db59e9c72bfd4c30d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:35:41.086 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:40 smithi172 bash[69260]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4 2024-04-23T18:35:41.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:41 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:41.092+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:41.354 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:41 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:41.265+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:41.354 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:41 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:41.265+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:41.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:41 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:41.056+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:41.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:41 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:41.056+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:41.437 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:41 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:41.364+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:41.437 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:41 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:41.364+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:41.437 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69278]: 2024-04-23 18:35:41.126076349 +0000 UTC m=+0.617099206 container remove 2b34095d97f5541e7492995cd6fff4dbddef837afa954db59e9c72bfd4c30d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T18:35:41.754 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:41 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:41.520+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:41.754 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69374]: 2024-04-23 18:35:41.348506218 +0000 UTC m=+0.018232522 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:41.754 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69374]: 2024-04-23 18:35:41.493159217 +0000 UTC m=+0.162885496 container create 8f43779c59ed37f2bdb2c8a4809cbf30f81a3d4e35cd46c3e846e0e3dd9a3f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T18:35:41.754 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69374]: 2024-04-23 18:35:41.688012736 +0000 UTC m=+0.357739021 container init 8f43779c59ed37f2bdb2c8a4809cbf30f81a3d4e35cd46c3e846e0e3dd9a3f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T18:35:41.754 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69374]: 2024-04-23 18:35:41.691318905 +0000 UTC m=+0.361045182 container start 8f43779c59ed37f2bdb2c8a4809cbf30f81a3d4e35cd46c3e846e0e3dd9a3f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T18:35:41.755 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69374]: 2024-04-23 18:35:41.762669979 +0000 UTC m=+0.432396258 container attach 8f43779c59ed37f2bdb2c8a4809cbf30f81a3d4e35cd46c3e846e0e3dd9a3f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:41.854 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:41 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:41.401+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:42.043 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:41 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:41.677+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:42.043 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:41 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:41.677+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:42.086 INFO:journalctl@ceph.osd.4.smithi172.stdout:Apr 23 18:35:41 smithi172 podman[69374]: 2024-04-23 18:35:41.9881253 +0000 UTC m=+0.657851581 container died 8f43779c59ed37f2bdb2c8a4809cbf30f81a3d4e35cd46c3e846e0e3dd9a3f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T18:35:42.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:42 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:42.051+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:42.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:42 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:42.051+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:42.378 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:42.311+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:42.378 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:42.311+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:42.378 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:42.129+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:42.378 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:42.129+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.409170+0000 front 2024-04-23T18:35:16.409190+0000 (oldest deadline 2024-04-23T18:35:41.109366+0000) 2024-04-23T18:35:42.582 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:42 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:42.474+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:42.582 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:42 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:42.398+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:42.583 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:42 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:42.398+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:42.613 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T18:35:42.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:42.652 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T18:35:42.652 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T18:35:42.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.5 2024-04-23T18:35:42.677 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:42.386+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:42.955 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:42 smithi173 systemd[1]: Stopping Ceph osd.5 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:43.043 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:42 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:42.693+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:43.043 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:42 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:42.693+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:43.352 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:42.963+0000 7fd9c38e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:43.352 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:42.963+0000 7fd9c38e1700 -1 osd.5 113 *** Got signal Terminated *** 2024-04-23T18:35:43.352 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:42 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:42.963+0000 7fd9c38e1700 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:43.352 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:43.086+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:43.352 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:43.086+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.409170+0000 front 2024-04-23T18:35:16.409190+0000 (oldest deadline 2024-04-23T18:35:41.109366+0000) 2024-04-23T18:35:43.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:43.051+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:43.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:43.051+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:43.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:43.403+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:43.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:43.360+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:43.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:43 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:43.360+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:43.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:43 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:43.432+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:43.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:43 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:43.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:43.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:43 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:43.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:43.996 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:43.737+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:43.996 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:43.737+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:44.339 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:44 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:44.041+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:44.339 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:44 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:44.041+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.409170+0000 front 2024-04-23T18:35:16.409190+0000 (oldest deadline 2024-04-23T18:35:41.109366+0000) 2024-04-23T18:35:44.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:44 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:44.004+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:44.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:44 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:44.004+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:44.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:44 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:44.425+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:44.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:44 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:44.347+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:44.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:44 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:44.347+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:44.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:44 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:44.474+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:44.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:44 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:44.403+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:44.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:44 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:44.403+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:44.999 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:44 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:44.726+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:45.000 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:44 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:44.726+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:45.353 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:45 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:45.057+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:45.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:45 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:45.057+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.409170+0000 front 2024-04-23T18:35:16.409190+0000 (oldest deadline 2024-04-23T18:35:41.109366+0000) 2024-04-23T18:35:45.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:45 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:45.007+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:45.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:45 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:45.007+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:45.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:45 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:45.435+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:45.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:45 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:45.367+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:45.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:45 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:45.367+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:45.854 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:45 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:45.446+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:45.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:45 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:45.388+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:45.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:45 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:45.388+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:46.003 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:45 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:45.721+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:46.003 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:45 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:45.721+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:46.003 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:45 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:45.721+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:46.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:46.097+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:46.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:46.097+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.409170+0000 front 2024-04-23T18:35:16.409190+0000 (oldest deadline 2024-04-23T18:35:41.109366+0000) 2024-04-23T18:35:46.355 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:46.346+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:46.355 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:46.346+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:46.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:46 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:46.011+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:46.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:46 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:46.011+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:46.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:46 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:46.011+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:46.589 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:46 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:46.427+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:46.590 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:46 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:46.427+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:46.591 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:46 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:46.320+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:46.591 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:46 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:46.320+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:46.591 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:46 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:46.320+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:46.853 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:46.448+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:46.854 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:46 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:46.448+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:47.000 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:46 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:46.733+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:47.000 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:46 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:46.733+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:47.000 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:46 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:46.733+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:47.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:47 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:47.102+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.307459+0000 front 2024-04-23T18:35:05.307446+0000 (oldest deadline 2024-04-23T18:35:30.007439+0000) 2024-04-23T18:35:47.354 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:47 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5[44720]: 2024-04-23T18:35:47.102+0000 7fd9bbecd700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.409170+0000 front 2024-04-23T18:35:16.409190+0000 (oldest deadline 2024-04-23T18:35:41.109366+0000) 2024-04-23T18:35:47.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:47.008+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:47.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:47.008+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:47.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:47.008+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:47.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:47 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:47.408+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:47.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:47 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:47.408+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:47.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:47 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:47.345+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:47.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:47 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:47.345+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:47.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:47 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:47.345+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:47.853 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:47 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:47.454+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:47.854 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:47 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:47.454+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:47.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:47 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:47.378+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:47.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:47 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:47.378+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:48.026 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:47.761+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:48.026 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:47.761+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:48.026 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:47.761+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:48.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:48.034+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:48.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:48.034+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:48.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:48.034+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:48.597 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:48 smithi173 podman[70992]: 2024-04-23 18:35:48.377351706 +0000 UTC m=+5.525042026 container died 97e1cbf6ac90583d2881cf5a45f656be634aee371f79cf47d269b50826a40fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:35:48.598 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:48 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:48.440+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:48.598 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:48 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:48.440+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:48.598 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:48 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:48.330+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:48.599 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:48 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:48.330+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:48.599 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:48 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:48.330+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:48.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:48 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:48.376+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:48.837 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:48 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:48.376+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:48.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:48 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:48.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:48.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:48 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:48.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:48.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:48 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:48.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:48.854 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:48 smithi173 podman[70992]: 2024-04-23 18:35:48.689153958 +0000 UTC m=+5.836844276 container cleanup 97e1cbf6ac90583d2881cf5a45f656be634aee371f79cf47d269b50826a40fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:35:48.854 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:48 smithi173 bash[70992]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5 2024-04-23T18:35:49.004 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:48.756+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:49.004 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:48.756+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:49.004 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:48 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:48.756+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:49.195 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:48 smithi173 podman[71007]: 2024-04-23 18:35:48.890331087 +0000 UTC m=+0.508953628 container remove 97e1cbf6ac90583d2881cf5a45f656be634aee371f79cf47d269b50826a40fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T18:35:49.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:49.012+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:49.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:49.012+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:49.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:49.012+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:49.502 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:49.433+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:49.502 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:49.433+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:49.503 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:49.306+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:49.503 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:49.306+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:49.503 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:49 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:49.306+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:49.504 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:49 smithi173 podman[71099]: 2024-04-23 18:35:49.107480642 +0000 UTC m=+0.018127587 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:49.504 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:49 smithi173 podman[71099]: 2024-04-23 18:35:49.208279255 +0000 UTC m=+0.118926229 container create 9f2e56ffbea1548c4d059df32a0b1ea0d4d07f2156da465d97d7c1ea8c7ecda2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T18:35:49.504 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:49 smithi173 podman[71099]: 2024-04-23 18:35:49.435880366 +0000 UTC m=+0.346527298 container init 9f2e56ffbea1548c4d059df32a0b1ea0d4d07f2156da465d97d7c1ea8c7ecda2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-23T18:35:49.504 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:49 smithi173 podman[71099]: 2024-04-23 18:35:49.438866098 +0000 UTC m=+0.349513022 container start 9f2e56ffbea1548c4d059df32a0b1ea0d4d07f2156da465d97d7c1ea8c7ecda2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T18:35:49.504 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:49 smithi173 podman[71099]: 2024-04-23 18:35:49.511056534 +0000 UTC m=+0.421703479 container attach 9f2e56ffbea1548c4d059df32a0b1ea0d4d07f2156da465d97d7c1ea8c7ecda2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:49.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:49 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:49.336+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:49.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:49 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:49.336+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:49.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:49 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:49.335+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:49.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:49 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:49.335+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:49.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:49 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:49.335+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:49.854 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:49 smithi173 podman[71099]: 2024-04-23 18:35:49.734459299 +0000 UTC m=+0.645106229 container died 9f2e56ffbea1548c4d059df32a0b1ea0d4d07f2156da465d97d7c1ea8c7ecda2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-23T18:35:49.976 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:49.733+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:49.976 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:49.733+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:49.976 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:49.733+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:50.254 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T18:35:50.317 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:50 smithi173 podman[71125]: 2024-04-23 18:35:50.240539154 +0000 UTC m=+0.500529982 container remove 9f2e56ffbea1548c4d059df32a0b1ea0d4d07f2156da465d97d7c1ea8c7ecda2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:35:50.318 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:50 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-23T18:35:50.318 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:50 smithi173 systemd[1]: Stopped Ceph osd.5 for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:35:50.318 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 23 18:35:50 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 3.167s CPU time. 2024-04-23T18:35:50.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:50.333 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T18:35:50.333 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-23T18:35:50.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.6 2024-04-23T18:35:50.361 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:49.984+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:50.361 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:49.984+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:50.361 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:49 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:49.984+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:50.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:50 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:50.316+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:50.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:50 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:50.316+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:50.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:50 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:50.341+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:50.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:50 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:50.341+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:50.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:50 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:50.341+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:50.603 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:50 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:50.480+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:50.603 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:50 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:50.480+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:50.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:50 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:50.353+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:50.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:50 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:50.353+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:50.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:50 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:50.353+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:50.619 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 systemd[1]: Stopping Ceph osd.6 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:50.970 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:50.677+0000 7f0a23b78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:50.970 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:50.677+0000 7f0a23b78700 -1 osd.6 113 *** Got signal Terminated *** 2024-04-23T18:35:50.971 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:50.677+0000 7f0a23b78700 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:50.971 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:50.705+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:50.971 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:50.705+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:50.971 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:50.705+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:51.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:50.978+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:51.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:50.978+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:51.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:50 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:50.978+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:51.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:51.340+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:51.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:51.340+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:51.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:51.331+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:51.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:51.331+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:51.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:51 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:51.331+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:51.854 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:51.513+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:51.854 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:51.513+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:51.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:51.366+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:51.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:51.366+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:51.855 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:51 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:51.366+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:51.937 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:51.669+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:51.937 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:51.669+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:51.937 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:51.669+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:52.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:51.945+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:52.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:51.945+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:52.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:51 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:51.945+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:52.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:52.376+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:52.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:52.376+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:52.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:52.330+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:52.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:52.330+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:52.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:52 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:52.330+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:52.603 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:52.481+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:52.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:52.481+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:52.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:52.337+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:52.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:52.337+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:52.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:52 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:52.337+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:52.908 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:52.657+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:52.908 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:52.657+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:52.908 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:52.657+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:52.909 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:52.657+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.489354+0000 front 2024-04-23T18:35:28.489426+0000 (oldest deadline 2024-04-23T18:35:52.589513+0000) 2024-04-23T18:35:53.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:52.916+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:53.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:52.916+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:53.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:52.916+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:53.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:53.501+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:53.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:53.501+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:53.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:53.349+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:53.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:53.349+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:53.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:53 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:53.349+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:53.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:53.352+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:53.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:53.352+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:53.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:53.352+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:53.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:53.348+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:53.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:53.348+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:53.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:53 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:53.348+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:53.898 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:53.651+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:53.898 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:53.651+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:53.899 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:53.651+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:53.899 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:53.651+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.489354+0000 front 2024-04-23T18:35:28.489426+0000 (oldest deadline 2024-04-23T18:35:52.589513+0000) 2024-04-23T18:35:54.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:53.906+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:54.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:53.906+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:54.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:53.906+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:54.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:53 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:53.906+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:35:54.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:54.315+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:54.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:54.315+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:54.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:54.315+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:54.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:54.342+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:54.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:54.342+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:54.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:54 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:54.342+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:54.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:54.528+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:54.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:54.528+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:54.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:54.309+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:54.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:54.309+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:54.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:54 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:54.309+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:55.119 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:54.686+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:54.686+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:54.686+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:54.686+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.489354+0000 front 2024-04-23T18:35:28.489426+0000 (oldest deadline 2024-04-23T18:35:52.589513+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:54.927+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:54.927+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:54.927+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:55.120 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:54 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:54.927+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:35:55.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:55.296+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:55.587 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:55.296+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:55.587 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:55.296+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:55.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:55.327+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:55.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:55.327+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:55.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:55 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:55.327+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:55.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:55.513+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:55.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:55.513+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:55.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:55.319+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:55.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:55.319+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:55.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:55.319+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:55.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:55 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:55.319+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:35:55.925 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:55.655+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.983253+0000 front 2024-04-23T18:35:05.983203+0000 (oldest deadline 2024-04-23T18:35:30.683405+0000) 2024-04-23T18:35:55.925 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:55.655+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.386149+0000 front 2024-04-23T18:35:16.386199+0000 (oldest deadline 2024-04-23T18:35:38.686300+0000) 2024-04-23T18:35:55.925 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:55.655+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.987696+0000 front 2024-04-23T18:35:20.987750+0000 (oldest deadline 2024-04-23T18:35:45.687799+0000) 2024-04-23T18:35:55.925 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6[55208]: 2024-04-23T18:35:55.655+0000 7f0a1c164700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.489354+0000 front 2024-04-23T18:35:28.489426+0000 (oldest deadline 2024-04-23T18:35:52.589513+0000) 2024-04-23T18:35:56.369 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:56 smithi037 podman[91322]: 2024-04-23 18:35:56.067733773 +0000 UTC m=+5.523084480 container died 9086879a7bcd2259526097a7c8ed654047005b154b45976cfa62f6bfce254fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T18:35:56.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:55.933+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:56.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:55.933+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:56.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:55.933+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:56.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:55 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:55.933+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:35:56.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:56.327+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:56.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:56.327+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:56.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:56.327+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:56.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:56.324+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:56.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:56.324+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:56.587 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:56 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:56.324+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:56.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:56.514+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:56.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:56.514+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:56.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:56.514+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:35:56.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:56.289+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:56.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:56.289+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:56.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:56.289+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:56.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:56 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:56.289+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:35:56.869 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:56 smithi037 podman[91322]: 2024-04-23 18:35:56.511085986 +0000 UTC m=+5.966436732 container cleanup 9086879a7bcd2259526097a7c8ed654047005b154b45976cfa62f6bfce254fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:35:56.869 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:56 smithi037 bash[91322]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6 2024-04-23T18:35:56.869 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:56 smithi037 podman[91339]: 2024-04-23 18:35:56.670389775 +0000 UTC m=+0.600389125 container remove 9086879a7bcd2259526097a7c8ed654047005b154b45976cfa62f6bfce254fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T18:35:57.181 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:57 smithi037 podman[91426]: 2024-04-23 18:35:56.989860838 +0000 UTC m=+0.018275447 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:35:57.181 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:57 smithi037 podman[91426]: 2024-04-23 18:35:57.087917869 +0000 UTC m=+0.116332485 container create bffc68914395a9ae4afd5667864bc3648092a44a32e721d08c746f6afe3242be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T18:35:57.181 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:56 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:56.938+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:57.181 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:56 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:56.938+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:57.182 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:56 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:56.938+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:57.182 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:56 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:56.938+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:35:57.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:57.313+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:57.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:57.313+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:57.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:57.313+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:57.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:57.313+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:35:57.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:57.529+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:57.605 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:57.529+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:57.605 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:57 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:57.529+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:35:57.619 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:57 smithi037 podman[91426]: 2024-04-23 18:35:57.340149702 +0000 UTC m=+0.368564287 container init bffc68914395a9ae4afd5667864bc3648092a44a32e721d08c746f6afe3242be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T18:35:57.619 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:57 smithi037 podman[91426]: 2024-04-23 18:35:57.343199668 +0000 UTC m=+0.371614245 container start bffc68914395a9ae4afd5667864bc3648092a44a32e721d08c746f6afe3242be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:35:57.620 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:57 smithi037 podman[91426]: 2024-04-23 18:35:57.423730795 +0000 UTC m=+0.452145385 container attach bffc68914395a9ae4afd5667864bc3648092a44a32e721d08c746f6afe3242be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:35:57.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:57.371+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:57.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:57.371+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:57.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:57.371+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:57.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:57.359+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:57.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:57.359+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:57.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:57.359+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:57.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:57 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:57.359+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:35:57.942 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 23 18:35:57 smithi037 podman[91426]: 2024-04-23 18:35:57.646317773 +0000 UTC m=+0.674732359 container died bffc68914395a9ae4afd5667864bc3648092a44a32e721d08c746f6afe3242be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:35:58.335 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:57 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:57.950+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:58.335 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:57 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:57.950+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:58.335 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:57 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:57.950+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:58.335 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:57 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:57.950+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:35:58.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.6.service' 2024-04-23T18:35:58.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:35:58.444 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-23T18:35:58.444 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-23T18:35:58.444 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.7 2024-04-23T18:35:58.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:58.488+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:35:58.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:58.488+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:35:58.604 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:58.488+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:35:58.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:58.355+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:35:58.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:58.355+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:35:58.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:58.355+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:35:58.605 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:58 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:58.355+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:35:58.761 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:58.375+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:35:58.761 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:58.375+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:35:58.761 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:58.375+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:35:58.761 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:58.375+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:35:58.761 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:58.388+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:58.762 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:58.388+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:58.762 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:58.388+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:58.762 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 systemd[1]: Stopping Ceph osd.7 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:35:58.762 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:58.768+0000 7ff537a9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:35:58.763 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:58.768+0000 7ff537a9b700 -1 osd.7 113 *** Got signal Terminated *** 2024-04-23T18:35:58.763 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:58 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:58.768+0000 7ff537a9b700 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:35:59.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:58 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:58.974+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:35:59.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:58 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:58.974+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:35:59.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:58 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:58.974+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:35:59.370 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:58 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:58.974+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:35:59.836 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:59.364+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:35:59.862 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:59.364+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:35:59.929 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:35:59.364+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:35:59.970 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:59.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:00.012 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:59.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:00.053 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:59.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:00.095 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:35:59 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:35:59.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:00.137 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:59.455+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:00.178 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:59.455+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:00.220 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:35:59.455+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:00.262 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:59.397+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:00.304 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:59.397+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:00.395 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:59.397+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:00.487 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:35:59 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:35:59.397+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:00.538 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:59 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:59.930+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:00.579 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:59 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:59.930+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:00.620 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:59 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:59.930+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:00.679 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:35:59 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:35:59.930+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:00.721 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:00.343+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:36:00.800 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:00.343+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:36:00.837 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:00.343+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:36:00.879 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:00.436+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:00.921 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:00.436+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:00.962 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:00.436+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:01.004 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:00 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:00.436+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:01.038 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:00.448+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:01.071 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:00.448+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:01.104 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:00.448+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:01.146 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:00.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:01.187 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:00.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:01.229 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:00.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:01.271 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:00 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:00.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:01.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:00.925+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:01.396 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:00.925+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:01.496 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:00.925+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:01.537 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:00 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:00.925+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:01.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:01.308+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:36:01.613 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:01.308+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:36:01.654 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:01.308+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:36:01.699 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:01.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:01.738 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:01.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:01.779 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:01.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:01.821 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:01.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:01.889 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:01.421+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:01.930 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:01.421+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:01.971 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:01.421+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:02.013 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:01.440+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:02.055 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:01.440+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:02.097 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:01.440+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:02.138 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:01 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:01.440+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:02.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:01 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:01.908+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:02.402 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:01 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:01.908+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:02.438 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:01 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:01.908+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:02.480 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:01 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:01.908+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:02.522 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:01 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:01.908+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:02.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:02.291+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:36:02.613 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:02.291+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:36:02.655 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:02.291+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:36:02.697 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:02.291+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.197463+0000 front 2024-04-23T18:35:39.197488+0000 (oldest deadline 2024-04-23T18:36:02.097595+0000) 2024-04-23T18:36:02.739 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:02.449+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:02.788 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:02.449+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:02.830 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:02.449+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:02.880 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:02.449+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:02.922 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:02 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:02.449+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:02.965 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:02.384+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:03.005 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:02.384+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:03.072 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:02.384+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:03.114 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:02.455+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:03.183 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:02.455+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:03.222 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:02.455+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:03.264 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:02 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:02.455+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:03.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:02 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:02.880+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:03.397 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:02 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:02.880+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:03.439 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:02 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:02.880+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:03.481 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:02 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:02.880+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:03.531 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:02 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:02.880+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:03.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:03.333+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:15.293274+0000 front 2024-04-23T18:35:15.293316+0000 (oldest deadline 2024-04-23T18:35:36.993497+0000) 2024-04-23T18:36:03.614 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:03.333+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:21.094798+0000 front 2024-04-23T18:35:21.094631+0000 (oldest deadline 2024-04-23T18:35:45.794490+0000) 2024-04-23T18:36:03.656 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:03.333+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:29.295614+0000 front 2024-04-23T18:35:29.295548+0000 (oldest deadline 2024-04-23T18:35:52.795619+0000) 2024-04-23T18:36:03.698 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7[50390]: 2024-04-23T18:36:03.333+0000 7ff530087700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.197463+0000 front 2024-04-23T18:35:39.197488+0000 (oldest deadline 2024-04-23T18:36:02.097595+0000) 2024-04-23T18:36:03.740 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:03.497+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:03.772 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:03.497+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:03.856 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:03.498+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:03.898 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:03.498+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:03.939 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:03 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:03.498+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:03.982 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:03.375+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:04.032 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:03.375+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:04.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:03.375+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:04.115 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:03.438+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:04.156 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:03.438+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:04.189 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:03.438+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:04.231 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:03 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:03.438+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:04.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:03.892+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:04.398 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:03.892+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:04.440 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:03.892+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:04.481 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:03.892+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:04.523 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:03 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:03.892+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:04.586 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:04 smithi172 podman[69479]: 2024-04-23 18:36:04.318658496 +0000 UTC m=+5.677917167 container died d5e9c9881eef36d9fda722206f9168b1f082cd52b2cb7305cbabe0ea508f2a13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-23T18:36:04.615 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:04 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:04.471+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:04.698 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:04 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:04.471+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:04.740 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:04 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:04.471+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:04.782 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:04 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:04.471+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:04.862 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:04 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:04.471+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:04.899 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:04.400+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:04.940 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:04.400+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:04.973 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:04.400+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:05.015 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:04 smithi172 podman[69479]: 2024-04-23 18:36:04.684586965 +0000 UTC m=+6.043845638 container cleanup d5e9c9881eef36d9fda722206f9168b1f082cd52b2cb7305cbabe0ea508f2a13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T18:36:05.074 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:04 smithi172 bash[69479]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7 2024-04-23T18:36:05.116 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:04.409+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:05.157 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:04.409+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:05.198 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:04.410+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:05.232 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:04 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:04.410+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:05.266 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:04 smithi172 podman[69508]: 2024-04-23 18:36:04.935739806 +0000 UTC m=+0.615916492 container remove d5e9c9881eef36d9fda722206f9168b1f082cd52b2cb7305cbabe0ea508f2a13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T18:36:05.369 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:04 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:04.879+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:05.398 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:04 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:04.879+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:05.449 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:04 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:04.879+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:05.482 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:04 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:04.879+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:05.515 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:04 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:04.879+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:05.549 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:05.463+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:05.620 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:05.463+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:05.657 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:05.463+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:05.699 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:05.463+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:05.732 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:05.463+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:05.766 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:05 smithi172 podman[69595]: 2024-04-23 18:36:05.228222223 +0000 UTC m=+0.098243391 container create 1b530297960137d541acd3332c771c29056da6fb292bfe4d131fdccec7119323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T18:36:05.846 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:05 smithi172 podman[69595]: 2024-04-23 18:36:05.148668576 +0000 UTC m=+0.018689776 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:36:05.882 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:05 smithi172 podman[69595]: 2024-04-23 18:36:05.422307227 +0000 UTC m=+0.292328403 container init 1b530297960137d541acd3332c771c29056da6fb292bfe4d131fdccec7119323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-23T18:36:05.916 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:05 smithi172 podman[69595]: 2024-04-23 18:36:05.425440813 +0000 UTC m=+0.295461983 container start 1b530297960137d541acd3332c771c29056da6fb292bfe4d131fdccec7119323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T18:36:05.950 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:05.381+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:06.007 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:05.381+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:06.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:05.381+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:06.149 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:05.381+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:06.199 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:05 smithi172 podman[69595]: 2024-04-23 18:36:05.497052111 +0000 UTC m=+0.367073299 container attach 1b530297960137d541acd3332c771c29056da6fb292bfe4d131fdccec7119323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T18:36:06.232 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:05 smithi172 podman[69595]: 2024-04-23 18:36:05.717494349 +0000 UTC m=+0.587515524 container died 1b530297960137d541acd3332c771c29056da6fb292bfe4d131fdccec7119323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:36:06.266 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:05.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:06.299 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:05.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:06.332 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:05.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:06.382 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:05 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:05.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:06.416 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:05 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:05.871+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:06.449 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:05 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:05.871+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:06.483 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:05 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:05.871+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:06.516 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:05 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:05.871+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:06.572 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:05 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:05.871+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:06.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.7.service' 2024-04-23T18:36:06.633 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:06 smithi172 podman[69622]: 2024-04-23 18:36:06.28399432 +0000 UTC m=+0.556779560 container remove 1b530297960137d541acd3332c771c29056da6fb292bfe4d131fdccec7119323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T18:36:06.667 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:06 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-23T18:36:06.708 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:06 smithi172 systemd[1]: Stopped Ceph osd.7 for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:36:06.754 INFO:journalctl@ceph.osd.7.smithi172.stdout:Apr 23 18:36:06 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.7.service: Consumed 1min 12.118s CPU time. 2024-04-23T18:36:06.833 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:06 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:06.472+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:06.866 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:06 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:06.472+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:06.899 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:06 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:06.472+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:06.940 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:06 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:06.472+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:06.975 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:06 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:06.472+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:07.017 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:06.434+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:07.059 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:06.434+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:07.100 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:06.434+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:07.133 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:06.434+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:07.168 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:06.434+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:07.208 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:06.398+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:07.263 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:06.398+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:07.300 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:06.398+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:07.333 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:06 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:06.398+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:07.367 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:06.844+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:07.400 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:06.844+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:07.450 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:06.844+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:07.492 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:06.844+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:07.525 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:06 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:06.844+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:07.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:07.592 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-23T18:36:07.625 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-23T18:36:07.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.8 2024-04-23T18:36:07.734 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:07.407+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:07.767 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:07.407+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:07.800 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:07.407+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:07.834 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:07.407+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:07.867 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:07.403+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:07.950 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:07.403+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:07.984 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:07.403+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:08.017 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:07.403+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:08.051 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:07 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:07.403+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:08.193 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:07.519+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:08.226 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:07.519+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:08.267 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:07.519+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:08.301 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:07.519+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:08.334 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:07 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:07.519+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:08.375 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:07 smithi173 systemd[1]: Stopping Ceph osd.8 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:08.410 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:07.881+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:08.456 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:07.881+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:08.508 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:07.881+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:08.551 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:07.881+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:08.584 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:07 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:07.881+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:08.618 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.018+0000 7feb18069700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:08.651 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.018+0000 7feb18069700 -1 osd.8 113 *** Got signal Terminated *** 2024-04-23T18:36:08.684 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.018+0000 7feb18069700 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:36:08.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:08 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:08.480+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:08.909 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:08 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:08.480+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:08.959 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:08 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:08.480+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:08.992 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:08 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:08.480+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:09.026 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:08 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:08.480+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:09.060 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.404+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:09.143 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.404+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:09.176 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.404+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:09.209 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:08.404+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:09.243 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:08.418+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:09.284 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:08.418+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:09.333 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:08.418+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:09.368 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:08.418+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:09.401 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:08 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:08.418+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:09.435 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:08.902+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:09.468 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:08.902+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:09.501 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:08.902+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:09.535 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:08.902+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:09.585 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:08 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:08.902+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:09.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:09.465+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:09.876 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:09.465+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:09.910 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:09.465+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:09.943 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:09.465+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:09.976 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:09.465+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:10.011 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:09.431+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:10.043 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:09.431+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:10.076 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:09.431+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:10.110 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:09.431+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:10.188 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:09.445+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:10.570 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:09.445+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:10.610 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:09.445+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:10.660 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:09.445+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:10.693 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:09 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:09.445+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:10.728 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:09.897+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:10.785 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:09.897+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:10.818 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:09.897+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:10.852 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:09.897+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:10.885 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:09 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:09.897+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:10.920 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:10.422+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:10.952 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:10.422+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:10.985 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:10.422+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:11.064 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:10.422+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:11.102 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:10 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:10.470+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:11.163 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:10 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:10.470+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:11.235 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:10 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:10.470+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:11.269 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:10 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:10.470+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:11.392 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:10 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:10.470+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:11.427 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:10.483+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:11.461 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:10.483+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:11.494 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:10.483+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:11.569 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:10.483+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:11.602 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:10 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:10.483+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:11.636 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:10.847+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:11.669 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:10.847+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:11.702 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:10.847+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:11.736 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:10.847+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:11.769 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:10 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:10.847+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:11.803 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:11.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:11.836 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:11.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:11.870 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:11.428+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:11.903 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:11.429+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:11.969 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:11.429+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:12.003 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:11.429+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:12.068 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:11.441+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:12.103 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:11.441+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:12.136 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:11.441+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:12.211 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:11.441+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:12.245 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:11.441+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:12.278 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:11 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:11.441+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:12.311 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:11.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:12.345 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:11.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:12.378 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:11.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:12.411 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:11.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:12.461 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:11.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:12.495 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:11 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:11.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:12.529 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:11.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:12.562 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:11.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:12.612 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:11 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:11.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:12.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:12 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:12.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:12.912 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:12 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:12.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:12.993 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:12 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:12.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:13.030 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:12 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:12.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:13.095 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:12 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:12.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:13.129 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:12 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:12.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:13.163 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:12.840+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:13.195 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:12.840+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:13.229 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:12.840+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:13.262 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:12.840+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:13.295 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:12 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:12.840+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:13.329 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:12.430+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.625147+0000 front 2024-04-23T18:35:14.625358+0000 (oldest deadline 2024-04-23T18:35:38.725345+0000) 2024-04-23T18:36:13.362 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:12.430+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.426504+0000 front 2024-04-23T18:35:20.426531+0000 (oldest deadline 2024-04-23T18:35:46.326667+0000) 2024-04-23T18:36:13.396 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:12.430+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.027780+0000 front 2024-04-23T18:35:31.027815+0000 (oldest deadline 2024-04-23T18:35:55.628210+0000) 2024-04-23T18:36:13.472 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:12.430+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.729369+0000 front 2024-04-23T18:35:39.729330+0000 (oldest deadline 2024-04-23T18:36:04.429326+0000) 2024-04-23T18:36:13.512 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8[51307]: 2024-04-23T18:36:12.430+0000 7feb1465d700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:47.730842+0000 front 2024-04-23T18:35:47.730820+0000 (oldest deadline 2024-04-23T18:36:11.830957+0000) 2024-04-23T18:36:13.546 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:12.518+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:13.624 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:12.518+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:13.662 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:12.518+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:13.726 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:12.518+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:13.779 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:12.518+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:13.812 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:12 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:12.518+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:13.847 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:13 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:13.420+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:13.879 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:13 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:13.420+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:13.913 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:13 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:13.420+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:13.946 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:13 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:13.420+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:13.979 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:13 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:13.420+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:14.013 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:13 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:13.420+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:14.046 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:13 smithi173 podman[71217]: 2024-04-23 18:36:13.52201924 +0000 UTC m=+5.613580775 container died b84a9ead4bfc2d9b41278a1cb1b81de648696515ce40043da58f2036ab7454f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T18:36:14.080 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:13 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:13.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:14.113 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:13 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:13.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:14.146 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:13 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:13.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:14.180 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:13 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:13.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:14.213 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:13 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:13.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:14.264 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:13 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:13.515+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:14.305 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:13.831+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:14.398 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:13.831+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:14.438 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:13.831+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:14.471 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:13.831+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:14.505 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:13.831+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:14.538 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:13 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:13.831+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:14.572 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:13 smithi173 podman[71217]: 2024-04-23 18:36:13.835692202 +0000 UTC m=+5.927253747 container cleanup b84a9ead4bfc2d9b41278a1cb1b81de648696515ce40043da58f2036ab7454f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:36:14.621 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:13 smithi173 bash[71217]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8 2024-04-23T18:36:14.655 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71236]: 2024-04-23 18:36:14.02854572 +0000 UTC m=+0.505399932 container remove b84a9ead4bfc2d9b41278a1cb1b81de648696515ce40043da58f2036ab7454f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-23T18:36:14.689 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71329]: 2024-04-23 18:36:14.437669023 +0000 UTC m=+0.098295671 container create fbf9e0065b776f0bae04da3aa11ea7fae1f7963200db59c634bd1994bef88ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:14.721 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71329]: 2024-04-23 18:36:14.357736062 +0000 UTC m=+0.018362731 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:36:14.755 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:14 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:14.564+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:14.788 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:14 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:14.564+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:14.822 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:14 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:14.564+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:14.855 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:14 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:14.564+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:14.888 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:14 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:14.564+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:14.940 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:14 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:14.564+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:15.006 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:14.389+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:15.072 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:14.389+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:15.113 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:14.389+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:15.147 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:14.389+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:15.180 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:14.389+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:15.214 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:14.389+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:15.247 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71329]: 2024-04-23 18:36:14.690553538 +0000 UTC m=+0.351180182 container init fbf9e0065b776f0bae04da3aa11ea7fae1f7963200db59c634bd1994bef88ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T18:36:15.280 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71329]: 2024-04-23 18:36:14.694034294 +0000 UTC m=+0.354660942 container start fbf9e0065b776f0bae04da3aa11ea7fae1f7963200db59c634bd1994bef88ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-04-23T18:36:15.314 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71329]: 2024-04-23 18:36:14.782149284 +0000 UTC m=+0.442775943 container attach fbf9e0065b776f0bae04da3aa11ea7fae1f7963200db59c634bd1994bef88ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-23T18:36:15.356 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:14.791+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:15.414 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:14.791+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:15.447 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:14.791+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:15.480 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:14.791+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:15.525 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:14.791+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:15.572 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:14 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:14.791+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:15.647 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:14 smithi173 podman[71329]: 2024-04-23 18:36:14.989532019 +0000 UTC m=+0.650158673 container died fbf9e0065b776f0bae04da3aa11ea7fae1f7963200db59c634bd1994bef88ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T18:36:15.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.8.service' 2024-04-23T18:36:15.801 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:15 smithi173 podman[71354]: 2024-04-23 18:36:15.527639463 +0000 UTC m=+0.532854914 container remove fbf9e0065b776f0bae04da3aa11ea7fae1f7963200db59c634bd1994bef88ee6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:36:15.897 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:15 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-23T18:36:15.931 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:15 smithi173 systemd[1]: Stopped Ceph osd.8 for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:36:15.964 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 23 18:36:15 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.8.service: Consumed 55.862s CPU time. 2024-04-23T18:36:15.998 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:15 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:15.579+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:16.031 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:15 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:15.579+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:16.064 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:15 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:15.579+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:16.097 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:15 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:15.579+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:16.131 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:15 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:15.579+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:16.164 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:15 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:15.579+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:16.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:16.231 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-23T18:36:16.264 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-23T18:36:16.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.9 2024-04-23T18:36:16.371 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:15 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:15.825+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:16.439 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:15 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:15.825+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:16.518 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:15 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:15.825+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:16.556 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:15 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:15.825+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:16.627 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:15 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:15.825+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:16.665 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:15 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:15.825+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:16.698 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:15 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:15.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:16.731 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:15 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:15.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:16.765 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:15 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:15.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:16.798 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:15 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:15.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:16.831 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:15 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:15.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:16.865 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:15 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:15.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:16.899 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:16 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:16.532+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:16.932 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:16 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:16.532+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:16.965 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:16 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:16.532+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:16.998 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:16 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:16.532+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:17.032 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:16 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:16.532+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:17.119 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:16 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:16.532+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:17.157 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:16 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:16.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:17.190 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:16 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:16.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:17.223 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:16 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:16.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:17.298 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:16 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:16.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:17.332 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:16 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:16.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:17.365 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:16 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:16.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:17.399 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 systemd[1]: Stopping Ceph osd.9 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:17.432 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.850+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:17.465 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.850+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:17.498 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.850+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:17.532 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.850+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:17.565 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.850+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:17.598 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.850+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:17.657 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.986+0000 7f423e041700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:17.690 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.986+0000 7f423e041700 -1 osd.9 113 *** Got signal Terminated *** 2024-04-23T18:36:17.740 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:16 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:16.986+0000 7f423e041700 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:36:17.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:17.866 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:17.907 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:17.940 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:17.974 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:18.007 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:18.041 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:17 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:17.417+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:18.074 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:17.552+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:18.107 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:17.552+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:18.141 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:17.552+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:18.174 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:17.552+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:18.207 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:17.552+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:18.249 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:17 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:17.552+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:18.283 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:17 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:17.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:18.316 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:17 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:17.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:18.366 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:17 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:17.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:18.399 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:17 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:17.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:18.451 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:17 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:17.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:18.491 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:17 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:17.807+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:18.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:18.899 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:18.952 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:18.991 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:19.024 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:19.058 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:19.091 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:18 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:18.410+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:19.125 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:19.158 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:19.191 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:19.224 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:19.258 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:19.291 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:19.358 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:18 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:18.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:53.473280+0000 front 2024-04-23T18:35:53.473234+0000 (oldest deadline 2024-04-23T18:36:18.173408+0000) 2024-04-23T18:36:19.398 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:18.540+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:19.479 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:18.540+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:19.523 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:18.540+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:19.589 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:18.540+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:19.625 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:18.540+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:19.718 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:18 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:18.540+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:19.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:19.877 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:19.908 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:19.941 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:19.975 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:20.008 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:20.042 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:19 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:19.414+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:20.076 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:19.502+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:20.133 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:19.502+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:20.167 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:19.502+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:20.200 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:19.502+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:20.233 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:19.502+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:20.290 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:19.502+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:20.331 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:20.372 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:20.408 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:20.442 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:20.475 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:20.509 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:20.542 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:19 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:19.822+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:53.473280+0000 front 2024-04-23T18:35:53.473234+0000 (oldest deadline 2024-04-23T18:36:18.173408+0000) 2024-04-23T18:36:20.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:20.900 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:20.934 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:20.967 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:21.017 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:21.050 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:21.086 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:20 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:20.425+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:21.137 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:21.196 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:21.250 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:21.313 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:21.373 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:21.420 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:21.470 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:20 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:20.857+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:53.473280+0000 front 2024-04-23T18:35:53.473234+0000 (oldest deadline 2024-04-23T18:36:18.173408+0000) 2024-04-23T18:36:21.520 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:20.460+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:21.570 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:20.460+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:21.620 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:20.460+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:21.670 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:20.460+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:21.722 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:20.460+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:21.770 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:20 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:20.460+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:21.837 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:21.887 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:21.937 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:21.987 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:22.037 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:22.087 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:22.151 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:21 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:21.428+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:22.196 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:07.763565+0000 front 2024-04-23T18:35:07.763942+0000 (oldest deadline 2024-04-23T18:35:29.463755+0000) 2024-04-23T18:36:22.245 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:16.167121+0000 front 2024-04-23T18:35:16.167020+0000 (oldest deadline 2024-04-23T18:35:39.067127+0000) 2024-04-23T18:36:22.295 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.168495+0000 front 2024-04-23T18:35:20.168460+0000 (oldest deadline 2024-04-23T18:35:45.468517+0000) 2024-04-23T18:36:22.345 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:30.169534+0000 front 2024-04-23T18:35:30.169484+0000 (oldest deadline 2024-04-23T18:35:53.069632+0000) 2024-04-23T18:36:22.395 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:37.071260+0000 front 2024-04-23T18:35:37.071281+0000 (oldest deadline 2024-04-23T18:36:01.771161+0000) 2024-04-23T18:36:22.445 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.272419+0000 front 2024-04-23T18:35:45.272452+0000 (oldest deadline 2024-04-23T18:36:13.472688+0000) 2024-04-23T18:36:22.495 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:21 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9[61820]: 2024-04-23T18:36:21.875+0000 7f423662d700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:53.473280+0000 front 2024-04-23T18:35:53.473234+0000 (oldest deadline 2024-04-23T18:36:18.173408+0000) 2024-04-23T18:36:22.545 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:21.498+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:22.595 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:21.498+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:22.662 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:21.498+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:22.739 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:21.498+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:22.829 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:21.498+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:22.879 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:21.498+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:22.930 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:22.979 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:23.029 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:23.098 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:23.154 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:23.204 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:23.254 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:22 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:22.418+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:23.304 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:22 smithi037 podman[91544]: 2024-04-23 18:36:22.360183609 +0000 UTC m=+5.452735477 container died 6866629c5bbd661b33d86bdd1d6309c1cdd5cfb4508c2d42ded284fb21b71384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T18:36:23.354 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:22 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:22.452+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:23.429 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:22 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:22.452+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:23.479 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:22 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:22.452+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:23.543 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:22 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:22.452+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:23.604 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:22 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:22.452+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:23.654 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:22 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:22.452+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:23.705 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:22 smithi037 podman[91544]: 2024-04-23 18:36:22.796115693 +0000 UTC m=+5.888667635 container cleanup 6866629c5bbd661b33d86bdd1d6309c1cdd5cfb4508c2d42ded284fb21b71384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:36:23.754 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:22 smithi037 bash[91544]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9 2024-04-23T18:36:23.804 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:22 smithi037 podman[91562]: 2024-04-23 18:36:22.988811136 +0000 UTC m=+0.631387226 container remove 6866629c5bbd661b33d86bdd1d6309c1cdd5cfb4508c2d42ded284fb21b71384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T18:36:23.854 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:23 smithi037 podman[91653]: 2024-04-23 18:36:23.323197003 +0000 UTC m=+0.018406014 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:36:23.904 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:23 smithi037 podman[91653]: 2024-04-23 18:36:23.431257754 +0000 UTC m=+0.126466765 container create abb0b6cb55772b9591e92c77b998cb1f0d022574f9f975486b80d6da5aa4b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-23T18:36:23.954 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:23 smithi037 podman[91653]: 2024-04-23 18:36:23.667056421 +0000 UTC m=+0.362265432 container init abb0b6cb55772b9591e92c77b998cb1f0d022574f9f975486b80d6da5aa4b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:36:24.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:23 smithi037 podman[91653]: 2024-04-23 18:36:23.67052144 +0000 UTC m=+0.365730462 container start abb0b6cb55772b9591e92c77b998cb1f0d022574f9f975486b80d6da5aa4b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-23T18:36:24.080 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:23.408+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:24.129 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:23.408+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:24.180 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:23.408+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:24.230 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:23.408+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:24.280 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:23.408+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:24.330 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:23 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:23.408+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:24.380 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:24.446 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:24.496 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:24.546 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:24.596 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:24.646 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:24.697 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:23 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:23.437+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:24.747 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:23 smithi037 podman[91653]: 2024-04-23 18:36:23.766958859 +0000 UTC m=+0.462167873 container attach abb0b6cb55772b9591e92c77b998cb1f0d022574f9f975486b80d6da5aa4b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:24.797 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:24.395+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:24.847 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:24.395+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:24.901 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:24.395+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:24.947 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:24.395+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:24.997 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:24.395+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:25.047 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:24 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:24.395+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:25.097 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:23 smithi037 podman[91653]: 2024-04-23 18:36:23.976324673 +0000 UTC m=+0.671533687 container died abb0b6cb55772b9591e92c77b998cb1f0d022574f9f975486b80d6da5aa4b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:36:25.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.9.service' 2024-04-23T18:36:25.197 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:24 smithi037 podman[91679]: 2024-04-23 18:36:24.5287918 +0000 UTC m=+0.547038493 container remove abb0b6cb55772b9591e92c77b998cb1f0d022574f9f975486b80d6da5aa4b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T18:36:25.289 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:24 smithi037 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.9.service: Deactivated successfully. 2024-04-23T18:36:25.339 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:24 smithi037 systemd[1]: Stopped Ceph osd.9 for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:36:25.388 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 23 18:36:24 smithi037 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.9.service: Consumed 51.669s CPU time. 2024-04-23T18:36:25.440 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:25.489 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:25.575 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:25.631 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:25.681 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:25.731 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:25.783 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:24 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:24.411+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:25.856 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:25.424+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:25.905 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:25.424+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:25.955 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:25.424+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:26.014 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:25.424+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:26.064 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:25.424+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:26.114 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:25 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:25.424+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:26.197 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:26.247 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:26.297 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:26.347 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:26.397 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:26.448 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:26.497 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:25 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:25.407+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:26.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:26.598 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-23T18:36:26.685 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-23T18:36:26.788 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.10 2024-04-23T18:36:26.848 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:26.898 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:26.948 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:26.998 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:27.048 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:27.114 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:27.164 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:26 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:26.442+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:27.215 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:27.265 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:27.315 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:27.365 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:27.415 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:27.465 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:27.515 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:26.449+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:27.566 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 systemd[1]: Stopping Ceph osd.10 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:27.615 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:27.679 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:27.724 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:27.774 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:27.824 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:27.874 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:27.949 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.416+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:27.999 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.509+0000 7f91304ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:28.049 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.509+0000 7f91304ed700 -1 osd.10 113 *** Got signal Terminated *** 2024-04-23T18:36:28.099 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:27 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:27.509+0000 7f91304ed700 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:36:28.184 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:28.232 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:28.282 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:28.332 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:28.382 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:28.432 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:28.482 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:27 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:27.482+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:28.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:28.924 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:28.974 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:29.024 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:29.074 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:29.149 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:29.199 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:28 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:28.382+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:29.250 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:29.299 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:29.349 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:29.400 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:29.449 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:29.499 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:29.549 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:28 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:28.503+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:29.737 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:29.799 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:29.850 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:29.899 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:29.959 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:30.008 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:30.058 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:29 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:29.392+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:30.125 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:30.175 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:30.225 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:30.275 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:30.325 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:30.375 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:30.533 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:29.542+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:30.756 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:30.792 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:30.842 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:30.891 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:30.942 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:30.991 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:31.047 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:30 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:30.440+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:31.092 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:31.142 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:31.225 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:31.275 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:31.325 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:31.375 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:31.434 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:30 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:30.513+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:31.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:31.875 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:31.951 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:32.001 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:32.042 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:32.084 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:32.126 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:31 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:31.455+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:32.168 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:32.259 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:32.304 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:32.346 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:32.409 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:32.451 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:32.493 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:31 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:31.492+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:32.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.673947+0000 front 2024-04-23T18:35:05.674038+0000 (oldest deadline 2024-04-23T18:35:29.173913+0000) 2024-04-23T18:36:32.868 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:13.075755+0000 front 2024-04-23T18:35:13.075733+0000 (oldest deadline 2024-04-23T18:35:38.975932+0000) 2024-04-23T18:36:32.901 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:20.077165+0000 front 2024-04-23T18:35:20.077088+0000 (oldest deadline 2024-04-23T18:35:45.377088+0000) 2024-04-23T18:36:32.934 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:31.679446+0000 front 2024-04-23T18:35:31.679350+0000 (oldest deadline 2024-04-23T18:35:56.379301+0000) 2024-04-23T18:36:32.968 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:38.580878+0000 front 2024-04-23T18:35:38.580944+0000 (oldest deadline 2024-04-23T18:36:01.481016+0000) 2024-04-23T18:36:33.001 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:44.982159+0000 front 2024-04-23T18:35:44.982178+0000 (oldest deadline 2024-04-23T18:36:10.882027+0000) 2024-04-23T18:36:33.035 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:32 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10[56980]: 2024-04-23T18:36:32.439+0000 7f9128ad9700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-23T18:35:51.383248+0000 front 2024-04-23T18:35:51.383193+0000 (oldest deadline 2024-04-23T18:36:16.683067+0000) 2024-04-23T18:36:33.069 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:33.101 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:33.135 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:33.168 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:33.203 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:33.245 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:33.310 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:32 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:32.490+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:33.352 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:33 smithi172 podman[69700]: 2024-04-23 18:36:33.066162491 +0000 UTC m=+5.652471735 container died fe9d2613696866d0be2abcbef7789f18cd0cdccea422f4247dc2081bcb5f5575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T18:36:33.836 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:33 smithi172 podman[69700]: 2024-04-23 18:36:33.384867626 +0000 UTC m=+5.971176876 container cleanup fe9d2613696866d0be2abcbef7789f18cd0cdccea422f4247dc2081bcb5f5575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-23T18:36:33.877 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:33 smithi172 bash[69700]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10 2024-04-23T18:36:33.910 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:33 smithi172 podman[69730]: 2024-04-23 18:36:33.535886496 +0000 UTC m=+0.462552171 container remove fe9d2613696866d0be2abcbef7789f18cd0cdccea422f4247dc2081bcb5f5575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-23T18:36:33.944 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:33.977 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:34.010 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:34.044 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:34.077 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:34.135 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:34.169 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:33 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:33.495+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:34.246 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:33 smithi172 podman[69822]: 2024-04-23 18:36:33.857130789 +0000 UTC m=+0.018268978 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:36:34.294 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:33 smithi172 podman[69822]: 2024-04-23 18:36:33.978316911 +0000 UTC m=+0.139455129 container create 35cda9a6083a21274934d6ae227b8dbcdf643f92095829e27c43d899df2fc8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T18:36:34.327 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:34 smithi172 podman[69822]: 2024-04-23 18:36:34.180874077 +0000 UTC m=+0.342012253 container init 35cda9a6083a21274934d6ae227b8dbcdf643f92095829e27c43d899df2fc8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-23T18:36:34.370 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:34 smithi172 podman[69822]: 2024-04-23 18:36:34.184107722 +0000 UTC m=+0.345245898 container start 35cda9a6083a21274934d6ae227b8dbcdf643f92095829e27c43d899df2fc8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:34.453 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:34 smithi172 podman[69822]: 2024-04-23 18:36:34.255471343 +0000 UTC m=+0.416609521 container attach 35cda9a6083a21274934d6ae227b8dbcdf643f92095829e27c43d899df2fc8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:36:34.586 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:34 smithi172 podman[69822]: 2024-04-23 18:36:34.472540028 +0000 UTC m=+0.633678206 container died 35cda9a6083a21274934d6ae227b8dbcdf643f92095829e27c43d899df2fc8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T18:36:34.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:34.894 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:34.928 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:34.961 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:34.994 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:35.028 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:35.061 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:35.094 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:34 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:34.467+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:35.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.10.service' 2024-04-23T18:36:35.162 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:35 smithi172 podman[69848]: 2024-04-23 18:36:35.00861603 +0000 UTC m=+0.530904924 container remove 35cda9a6083a21274934d6ae227b8dbcdf643f92095829e27c43d899df2fc8b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:36:35.195 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:35 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-23T18:36:35.228 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:35 smithi172 systemd[1]: Stopped Ceph osd.10 for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:36:35.261 INFO:journalctl@ceph.osd.10.smithi172.stdout:Apr 23 18:36:35 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.10.service: Consumed 54.509s CPU time. 2024-04-23T18:36:35.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:35.462 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-23T18:36:35.495 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-23T18:36:35.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.11 2024-04-23T18:36:35.804 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:35.872 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:35.921 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:35.970 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:36.070 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:36.120 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:36.154 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:36.187 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.478+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:36.222 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 systemd[1]: Stopping Ceph osd.11 for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:36.255 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.871+0000 7f8179dbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:36.287 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.871+0000 7f8179dbf700 -1 osd.11 113 *** Got signal Terminated *** 2024-04-23T18:36:36.362 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:35 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:35.871+0000 7f8179dbf700 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T18:36:36.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:36.879 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:36.964 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:36.996 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:37.055 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:37.088 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:37.121 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:37.155 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:36 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:36.501+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:37.853 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:37.905 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:37.939 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:37.980 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:38.022 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:38.055 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:38.088 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:38.122 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:37 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:37.461+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:38.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:38.939 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:38.989 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:39.082 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:39.123 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:39.173 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:39.223 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:39.273 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:38 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:38.427+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:39.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:39.898 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:39.948 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:39.998 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:40.052 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:40.098 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:40.149 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:40.198 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:39 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:39.407+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:40.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-23T18:35:05.466541+0000 front 2024-04-23T18:35:05.466526+0000 (oldest deadline 2024-04-23T18:35:30.766325+0000) 2024-04-23T18:36:41.021 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6804 osd.1 since back 2024-04-23T18:35:14.768159+0000 front 2024-04-23T18:35:14.768289+0000 (oldest deadline 2024-04-23T18:35:38.268058+0000) 2024-04-23T18:36:41.224 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-23T18:35:23.469634+0000 front 2024-04-23T18:35:23.469679+0000 (oldest deadline 2024-04-23T18:35:48.169800+0000) 2024-04-23T18:36:41.274 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-23T18:35:28.671054+0000 front 2024-04-23T18:35:28.671091+0000 (oldest deadline 2024-04-23T18:35:54.571018+0000) 2024-04-23T18:36:41.362 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6812 osd.4 since back 2024-04-23T18:35:39.172624+0000 front 2024-04-23T18:35:39.172510+0000 (oldest deadline 2024-04-23T18:36:05.572910+0000) 2024-04-23T18:36:41.399 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-23T18:35:45.573245+0000 front 2024-04-23T18:35:45.573290+0000 (oldest deadline 2024-04-23T18:36:10.873360+0000) 2024-04-23T18:36:41.449 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.172:6820 osd.7 since back 2024-04-23T18:36:00.175362+0000 front 2024-04-23T18:36:00.175452+0000 (oldest deadline 2024-04-23T18:36:26.075364+0000) 2024-04-23T18:36:41.499 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:40 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11[58312]: 2024-04-23T18:36:40.411+0000 7f81763b3700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-23T18:36:11.976088+0000 front 2024-04-23T18:36:11.976060+0000 (oldest deadline 2024-04-23T18:36:34.276053+0000) 2024-04-23T18:36:41.596 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:41 smithi173 podman[71446]: 2024-04-23 18:36:41.278844194 +0000 UTC m=+5.512298223 container died 4c97afb0e8fe43dc4b293be1e8ead45b8f891d064d37ff9e0831fe1389069213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T18:36:41.854 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:41 smithi173 podman[71446]: 2024-04-23 18:36:41.604868228 +0000 UTC m=+5.838322245 container cleanup 4c97afb0e8fe43dc4b293be1e8ead45b8f891d064d37ff9e0831fe1389069213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T18:36:41.891 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:41 smithi173 bash[71446]: ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11 2024-04-23T18:36:41.941 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:41 smithi173 podman[71465]: 2024-04-23 18:36:41.789504244 +0000 UTC m=+0.512649513 container remove 4c97afb0e8fe43dc4b293be1e8ead45b8f891d064d37ff9e0831fe1389069213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-23T18:36:42.484 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:42 smithi173 podman[71555]: 2024-04-23 18:36:42.127172297 +0000 UTC m=+0.018067671 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:36:42.524 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:42 smithi173 podman[71555]: 2024-04-23 18:36:42.232141939 +0000 UTC m=+0.123037297 container create b163b019acb2f5dca3c1c9eb92f56b294ff34a6ca7ccde3fa382d842dd593dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:42.574 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:42 smithi173 podman[71555]: 2024-04-23 18:36:42.418006261 +0000 UTC m=+0.308901620 container init b163b019acb2f5dca3c1c9eb92f56b294ff34a6ca7ccde3fa382d842dd593dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T18:36:42.624 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:42 smithi173 podman[71555]: 2024-04-23 18:36:42.421179815 +0000 UTC m=+0.312075177 container start b163b019acb2f5dca3c1c9eb92f56b294ff34a6ca7ccde3fa382d842dd593dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T18:36:42.674 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:42 smithi173 podman[71555]: 2024-04-23 18:36:42.49301096 +0000 UTC m=+0.383906320 container attach b163b019acb2f5dca3c1c9eb92f56b294ff34a6ca7ccde3fa382d842dd593dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:36:42.853 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:42 smithi173 podman[71555]: 2024-04-23 18:36:42.709125503 +0000 UTC m=+0.600020861 container died b163b019acb2f5dca3c1c9eb92f56b294ff34a6ca7ccde3fa382d842dd593dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T18:36:43.245 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 23 18:36:43 smithi173 podman[71581]: 2024-04-23 18:36:43.240051406 +0000 UTC m=+0.526099107 container remove b163b019acb2f5dca3c1c9eb92f56b294ff34a6ca7ccde3fa382d842dd593dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:36:43.343 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@osd.11.service' 2024-04-23T18:36:43.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:43.468 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-23T18:36:43.517 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-23T18:36:43.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.a 2024-04-23T18:36:43.931 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:36:43 smithi037 systemd[1]: Stopping Ceph mds.a for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:44.210 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:36:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-a[68617]: 2024-04-23T18:36:43.939+0000 7f0aec984700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:44.242 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:36:43 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-a[68617]: 2024-04-23T18:36:43.939+0000 7f0aec984700 -1 mds.a *** got signal Terminated *** 2024-04-23T18:36:46.113 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:36:45 smithi037 podman[91770]: 2024-04-23 18:36:45.844337081 +0000 UTC m=+2.016890857 container died cb1eb13113c16daf020a9cbef0e7f6cceacc13789205e6fc3d9fa1ef58717b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:46.168 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:36:46 smithi037 podman[91770]: 2024-04-23 18:36:46.12219342 +0000 UTC m=+2.294747202 container cleanup cb1eb13113c16daf020a9cbef0e7f6cceacc13789205e6fc3d9fa1ef58717b73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-23T18:36:46.369 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 23 18:36:46 smithi037 bash[91770]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-a 2024-04-23T18:36:46.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.a.service' 2024-04-23T18:36:46.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:46.726 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-23T18:36:46.760 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-23T18:36:46.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.d 2024-04-23T18:36:47.174 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:36:46 smithi037 systemd[1]: Stopping Ceph mds.d for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:47.619 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:36:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d[67655]: 2024-04-23T18:36:47.182+0000 7f4f9d7bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:47.660 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:36:47 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d[67655]: 2024-04-23T18:36:47.182+0000 7f4f9d7bf700 -1 mds.d *** got signal Terminated *** 2024-04-23T18:36:51.483 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:36:51 smithi037 podman[91901]: 2024-04-23 18:36:51.190703208 +0000 UTC m=+4.102587931 container died 3fb45d89b5314b6188429aa39df58260b1ebd088250c3ab6f7268c2f3cf45a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-23T18:36:51.866 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:36:51 smithi037 podman[91901]: 2024-04-23 18:36:51.565936237 +0000 UTC m=+4.477820948 container cleanup 3fb45d89b5314b6188429aa39df58260b1ebd088250c3ab6f7268c2f3cf45a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:51.912 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 23 18:36:51 smithi037 bash[91901]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-d 2024-04-23T18:36:51.998 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.d.service' 2024-04-23T18:36:52.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:52.106 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-23T18:36:52.156 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-23T18:36:52.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.g 2024-04-23T18:36:52.561 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:36:52 smithi037 systemd[1]: Stopping Ceph mds.g for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:52.869 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:36:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g[68252]: 2024-04-23T18:36:52.569+0000 7fb385777700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:52.948 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:36:52 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g[68252]: 2024-04-23T18:36:52.569+0000 7fb385777700 -1 mds.g *** got signal Terminated *** 2024-04-23T18:36:55.311 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:36:55 smithi037 podman[92020]: 2024-04-23 18:36:55.053835235 +0000 UTC m=+2.609623495 container died a289d6c64445d5520c8938fe544cfa3a5fa2e563f3245fbdb88470919d9b75b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-23T18:36:55.619 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:36:55 smithi037 podman[92020]: 2024-04-23 18:36:55.411662857 +0000 UTC m=+2.967451116 container cleanup a289d6c64445d5520c8938fe544cfa3a5fa2e563f3245fbdb88470919d9b75b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:36:55.738 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 23 18:36:55 smithi037 bash[92020]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-g 2024-04-23T18:36:55.786 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.g.service' 2024-04-23T18:36:55.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:36:55.874 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-23T18:36:55.908 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-23T18:36:55.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.j 2024-04-23T18:36:56.257 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:36:56 smithi037 systemd[1]: Stopping Ceph mds.j for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:36:56.619 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:36:56 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j[67303]: 2024-04-23T18:36:56.265+0000 7f5e86b24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:36:56.674 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:36:56 smithi037 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j[67303]: 2024-04-23T18:36:56.265+0000 7f5e86b24700 -1 mds.j *** got signal Terminated *** 2024-04-23T18:37:00.619 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:37:00 smithi037 podman[92141]: 2024-04-23 18:37:00.280123581 +0000 UTC m=+4.114123698 container died f7505dc57dfa70fa8751643e58e0e69694e9194d6a6286e4e5a0ef52778ce8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS) 2024-04-23T18:37:00.990 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:37:00 smithi037 podman[92141]: 2024-04-23 18:37:00.756754731 +0000 UTC m=+4.590754847 container cleanup f7505dc57dfa70fa8751643e58e0e69694e9194d6a6286e4e5a0ef52778ce8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T18:37:01.074 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 23 18:37:00 smithi037 bash[92141]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-j 2024-04-23T18:37:01.108 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.j.service' 2024-04-23T18:37:01.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:01.207 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-23T18:37:01.241 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-23T18:37:01.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.b 2024-04-23T18:37:01.587 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:37:01 smithi172 systemd[1]: Stopping Ceph mds.b for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:01.845 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:37:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: 2024-04-23T18:37:01.616+0000 7f08fe3a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:01.874 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:37:01 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b[62210]: 2024-04-23T18:37:01.616+0000 7f08fe3a8700 -1 mds.b *** got signal Terminated *** 2024-04-23T18:37:04.883 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:37:04 smithi172 podman[69928]: 2024-04-23 18:37:04.612544932 +0000 UTC m=+3.098398964 container died 7aabb44825f8180487b0b211fb2ebb38af44d90ae6e5d33c8020bb5278cc14c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:37:05.233 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:37:05 smithi172 podman[69928]: 2024-04-23 18:37:04.999980279 +0000 UTC m=+3.485834310 container cleanup 7aabb44825f8180487b0b211fb2ebb38af44d90ae6e5d33c8020bb5278cc14c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:37:05.263 INFO:journalctl@ceph.mds.b.smithi172.stdout:Apr 23 18:37:05 smithi172 bash[69928]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-b 2024-04-23T18:37:05.374 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.b.service' 2024-04-23T18:37:05.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:05.457 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-23T18:37:05.491 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-23T18:37:05.524 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.e 2024-04-23T18:37:05.836 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:37:05 smithi172 systemd[1]: Stopping Ceph mds.e for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:06.336 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:37:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e[61583]: 2024-04-23T18:37:05.845+0000 7f71794cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:06.382 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:37:05 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e[61583]: 2024-04-23T18:37:05.845+0000 7f71794cc700 -1 mds.e *** got signal Terminated *** 2024-04-23T18:37:08.886 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:37:08 smithi172 podman[70064]: 2024-04-23 18:37:08.611886612 +0000 UTC m=+2.875979110 container died f9baf81f3881ee8b1394373b6a3d49d5dd040eb72bb11034d57b3a90b603c1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:37:09.170 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:37:08 smithi172 podman[70064]: 2024-04-23 18:37:08.894991822 +0000 UTC m=+3.159084316 container cleanup f9baf81f3881ee8b1394373b6a3d49d5dd040eb72bb11034d57b3a90b603c1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:37:09.190 INFO:journalctl@ceph.mds.e.smithi172.stdout:Apr 23 18:37:08 smithi172 bash[70064]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-e 2024-04-23T18:37:09.320 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.e.service' 2024-04-23T18:37:09.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:09.399 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-23T18:37:09.432 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-23T18:37:09.466 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.h 2024-04-23T18:37:09.781 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:37:09 smithi172 systemd[1]: Stopping Ceph mds.h for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:10.086 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:37:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h[61890]: 2024-04-23T18:37:09.789+0000 7f869c697700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:10.124 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:37:09 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h[61890]: 2024-04-23T18:37:09.789+0000 7f869c697700 -1 mds.h *** got signal Terminated *** 2024-04-23T18:37:12.848 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:37:12 smithi172 podman[70185]: 2024-04-23 18:37:12.606540255 +0000 UTC m=+2.922401483 container died 96c8aaefce8d05593e2c75f22c76a2ff5ed506417f1ac4ece76e3ccc2713e801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T18:37:13.399 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:37:13 smithi172 podman[70185]: 2024-04-23 18:37:13.182001616 +0000 UTC m=+3.497862843 container cleanup 96c8aaefce8d05593e2c75f22c76a2ff5ed506417f1ac4ece76e3ccc2713e801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:37:13.432 INFO:journalctl@ceph.mds.h.smithi172.stdout:Apr 23 18:37:13 smithi172 bash[70185]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-h 2024-04-23T18:37:13.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.h.service' 2024-04-23T18:37:13.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:13.632 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-23T18:37:13.665 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-23T18:37:13.721 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.k 2024-04-23T18:37:14.043 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:13 smithi172 systemd[1]: Stopping Ceph mds.k for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:14.336 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k[61260]: 2024-04-23T18:37:14.052+0000 7f10fffcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:14.382 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:14 smithi172 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k[61260]: 2024-04-23T18:37:14.052+0000 7f10fffcd700 -1 mds.k *** got signal Terminated *** 2024-04-23T18:37:18.154 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:17 smithi172 podman[70305]: 2024-04-23 18:37:17.881053208 +0000 UTC m=+3.939659014 container died ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T18:37:18.409 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:18 smithi172 podman[70305]: 2024-04-23 18:37:18.40190707 +0000 UTC m=+4.460512875 container cleanup ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:37:18.441 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:18 smithi172 bash[70305]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k 2024-04-23T18:37:18.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.k.service' 2024-04-23T18:37:18.877 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:18 smithi172 podman[70319]: 2024-04-23 18:37:18.602933024 +0000 UTC m=+0.719306037 container remove ca2429afcee75918b6267b16b06200c96e0261380c398073c68fb320d5722301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:37:18.956 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:18 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-23T18:37:18.990 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:18 smithi172 systemd[1]: Stopped Ceph mds.k for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:37:19.024 INFO:journalctl@ceph.mds.k.smithi172.stdout:Apr 23 18:37:18 smithi172 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.127s CPU time. 2024-04-23T18:37:19.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:19.099 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-23T18:37:19.132 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-23T18:37:19.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.c 2024-04-23T18:37:19.603 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:37:19 smithi173 systemd[1]: Stopping Ceph mds.c for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:20.103 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:37:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: 2024-04-23T18:37:19.621+0000 7f0208e37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:20.140 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:37:19 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c[63200]: 2024-04-23T18:37:19.621+0000 7f0208e37700 -1 mds.c *** got signal Terminated *** 2024-04-23T18:37:20.853 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:37:20 smithi173 podman[71660]: 2024-04-23 18:37:20.605692883 +0000 UTC m=+1.163751080 container died edc689ee73c259925de726fe4c30a9bc0afd75605321e00b1f2260dcc504b459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T18:37:21.248 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:37:20 smithi173 podman[71660]: 2024-04-23 18:37:20.963658571 +0000 UTC m=+1.521716768 container cleanup edc689ee73c259925de726fe4c30a9bc0afd75605321e00b1f2260dcc504b459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:37:21.291 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 23 18:37:20 smithi173 bash[71660]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-c 2024-04-23T18:37:21.372 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.c.service' 2024-04-23T18:37:21.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:21.458 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-23T18:37:21.497 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-23T18:37:21.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.f 2024-04-23T18:37:21.851 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:21 smithi173 systemd[1]: Stopping Ceph mds.f for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:22.103 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f[62886]: 2024-04-23T18:37:21.858+0000 7fe741a45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:23.608 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:21 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f[62886]: 2024-04-23T18:37:21.858+0000 7fe741a45700 -1 mds.f *** got signal Terminated *** 2024-04-23T18:37:25.052 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:24 smithi173 podman[71790]: 2024-04-23 18:37:24.628081884 +0000 UTC m=+2.865476964 container died d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:37:25.384 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:25 smithi173 podman[71790]: 2024-04-23 18:37:25.14299488 +0000 UTC m=+3.380389955 container cleanup d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:37:25.483 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:25 smithi173 bash[71790]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f 2024-04-23T18:37:25.562 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.f.service' 2024-04-23T18:37:25.603 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:25 smithi173 podman[71805]: 2024-04-23 18:37:25.35218759 +0000 UTC m=+0.724895157 container remove d3080db2a86871553eb5e72b1e1711113b3568d8ee3a2708a5696abaddf09da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T18:37:25.625 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:25 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-23T18:37:25.658 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:25 smithi173 systemd[1]: Stopped Ceph mds.f for af48a038-019c-11ef-bc93-c7b262605968. 2024-04-23T18:37:25.691 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 23 18:37:25 smithi173 systemd[1]: ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.f.service: Consumed 9.633s CPU time. 2024-04-23T18:37:25.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:25.833 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-23T18:37:25.866 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-23T18:37:25.900 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.i 2024-04-23T18:37:26.205 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:37:25 smithi173 systemd[1]: Stopping Ceph mds.i for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:26.603 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:37:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i[63509]: 2024-04-23T18:37:26.212+0000 7f695de0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:26.650 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:37:26 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i[63509]: 2024-04-23T18:37:26.212+0000 7f695de0c700 -1 mds.i *** got signal Terminated *** 2024-04-23T18:37:28.888 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:37:28 smithi173 podman[71911]: 2024-04-23 18:37:28.621389529 +0000 UTC m=+2.502892591 container died f395a5ca45a63f6ed36ad858f2d62841158dc30b179a9ab8b1765ceb3ba755ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T18:37:29.239 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:37:28 smithi173 podman[71911]: 2024-04-23 18:37:28.99637123 +0000 UTC m=+2.877874331 container cleanup f395a5ca45a63f6ed36ad858f2d62841158dc30b179a9ab8b1765ceb3ba755ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T18:37:29.274 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 23 18:37:28 smithi173 bash[71911]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-i 2024-04-23T18:37:29.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.i.service' 2024-04-23T18:37:29.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:29.441 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-23T18:37:29.475 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-23T18:37:29.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.l 2024-04-23T18:37:29.824 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:29 smithi173 systemd[1]: Stopping Ceph mds.l for af48a038-019c-11ef-bc93-c7b262605968... 2024-04-23T18:37:30.103 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l[62581]: 2024-04-23T18:37:29.832+0000 7f196d315700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:37:30.133 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:29 smithi173 ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l[62581]: 2024-04-23T18:37:29.832+0000 7f196d315700 -1 mds.l *** got signal Terminated *** 2024-04-23T18:37:31.329 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:31 smithi173 podman[72031]: 2024-04-23 18:37:31.011064963 +0000 UTC m=+1.285243208 container died ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-23T18:37:31.603 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:31 smithi173 podman[72031]: 2024-04-23 18:37:31.342147863 +0000 UTC m=+1.616326107 container cleanup ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T18:37:31.625 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:31 smithi173 bash[72031]: ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l 2024-04-23T18:37:31.658 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 23 18:37:31 smithi173 podman[72045]: 2024-04-23 18:37:31.509744819 +0000 UTC m=+0.498120662 container remove ec75cb689fbf6644a77e3286249481c14854727a354433353064a5a49aa12318 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-af48a038-019c-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:37:31.746 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af48a038-019c-11ef-bc93-c7b262605968@mds.l.service' 2024-04-23T18:37:31.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T18:37:31.841 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-23T18:37:31.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af48a038-019c-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:37:32.175 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:37:34.727 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af48a038-019c-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:37:34.955 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:37:37.416 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af48a038-019c-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T18:37:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:37:39.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:37:40.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:37:40.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:37:40.142 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T18:37:40.167 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi037/crash 2024-04-23T18:37:40.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:37:40.275 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:37:40.351 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:37:40.384 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi172/crash 2024-04-23T18:37:40.449 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:37:40.546 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:37:40.566 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:37:40.602 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi173/crash 2024-04-23T18:37:40.633 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash -- . 2024-04-23T18:37:40.709 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/af48a038-019c-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T18:37:40.733 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-04-23T18:37:40.773 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T18:37:40.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T18:37:40.950 INFO:tasks.cephadm:Compressing logs... 2024-04-23T18:37:40.974 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:37:41.020 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:37:41.057 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-23T18:37:41.083 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:37:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:37:41.149 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-23T18:37:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-23T18:37:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T18:37:41.250 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T18:37:41.283 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-23T18:37:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-23T18:37:41.351 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-23T18:37:41.383 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.a.log: 87.2% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:37:41.417 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:37:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:37:41.500 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-23T18:37:41.553 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-23T18:37:41.592 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-23T18:37:41.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:37:41.662 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.a.log: 84.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:37:41.692 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-23T18:37:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-23T18:37:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-23T18:37:41.793 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T18:37:41.829 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-23T18:37:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:37:41.892 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:37:41.926 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:37:41.959 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-23T18:37:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:37:42.026 INFO:teuthology.orchestra.run.smithi172.stderr:: No such file or directory 2024-04-23T18:37:42.084 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:37:42.117 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-23T18:37:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-23T18:37:42.218 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T18:37:42.297 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-23T18:37:42.334 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-23T18:37:42.367 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-23T18:37:42.401 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-23T18:37:42.434 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:37:42.468 INFO:teuthology.orchestra.run.smithi172.stderr: 87.2% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:37:42.501 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:37:42.534 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.10.log: -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-23T18:37:42.568 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:37:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:37:42.634 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-23T18:37:42.668 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-23T18:37:42.718 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-23T18:37:42.751 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:37:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:37:42.836 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:37:42.886 INFO:teuthology.orchestra.run.smithi172.stderr: 94.5% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-23T18:37:42.919 INFO:teuthology.orchestra.run.smithi172.stderr: 91.8% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-23T18:37:42.954 INFO:teuthology.orchestra.run.smithi037.stderr: 90.3% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-23T18:37:42.985 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log 2024-04-23T18:37:43.018 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-23T18:37:43.051 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T18:37:43.085 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-23T18:37:43.118 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T18:37:43.151 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T18:37:43.185 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-23T18:37:43.218 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-23T18:37:43.252 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-23T18:37:43.310 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-23T18:37:43.343 INFO:teuthology.orchestra.run.smithi173.stderr: 87.2% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T18:37:43.385 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T18:37:43.418 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T18:37:43.460 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-23T18:37:43.493 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T18:37:43.527 INFO:teuthology.orchestra.run.smithi173.stderr: 92.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-23T18:37:43.560 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-23T18:37:43.594 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T18:37:43.627 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.8.log: /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-23T18:37:43.660 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T18:37:43.693 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-23T18:37:43.727 INFO:teuthology.orchestra.run.smithi173.stderr: 91.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T18:37:43.760 INFO:teuthology.orchestra.run.smithi173.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T18:37:43.794 INFO:teuthology.orchestra.run.smithi173.stderr: 92.7% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-23T18:37:43.828 INFO:teuthology.orchestra.run.smithi173.stderr: 95.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-23T18:37:44.864 INFO:teuthology.orchestra.run.smithi173.stderr: 95.8% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-23T18:37:45.114 INFO:teuthology.orchestra.run.smithi037.stderr: 94.3% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-23T18:37:45.179 INFO:teuthology.orchestra.run.smithi037.stderr: 93.9% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T18:37:45.700 INFO:teuthology.orchestra.run.smithi037.stderr: 94.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-23T18:37:45.938 INFO:teuthology.orchestra.run.smithi172.stderr: 95.6% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-23T18:37:46.125 INFO:teuthology.orchestra.run.smithi172.stderr: 94.3% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T18:37:46.406 INFO:teuthology.orchestra.run.smithi037.stderr: 94.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T18:37:46.495 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-23T18:37:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m5.374s 2024-04-23T18:37:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m18.905s 2024-04-23T18:37:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.658s 2024-04-23T18:37:46.628 INFO:teuthology.orchestra.run.smithi173.stderr: 94.2% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-23T18:37:46.901 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T18:37:47.095 INFO:teuthology.orchestra.run.smithi173.stderr: 94.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-23T18:37:47.427 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-23T18:37:47.563 INFO:teuthology.orchestra.run.smithi173.stderr: 94.1% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T18:37:48.229 INFO:teuthology.orchestra.run.smithi173.stderr: 94.3% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T18:37:48.318 INFO:teuthology.orchestra.run.smithi172.stderr: 94.2% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-23T18:38:08.349 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-23T18:38:22.514 INFO:teuthology.orchestra.run.smithi173.stderr: 94.0% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-23T18:38:22.516 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-23T18:38:22.517 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m40.671s 2024-04-23T18:38:22.517 INFO:teuthology.orchestra.run.smithi173.stderr:user 1m6.143s 2024-04-23T18:38:22.517 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m2.110s 2024-04-23T18:38:57.784 INFO:teuthology.orchestra.run.smithi172.stderr: 93.6% -- replaced with /var/log/ceph/af48a038-019c-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-23T18:38:57.785 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-23T18:38:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:real 1m16.718s 2024-04-23T18:38:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:user 2m9.401s 2024-04-23T18:38:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m4.176s 2024-04-23T18:38:57.787 INFO:tasks.cephadm:Archiving logs... 2024-04-23T18:38:57.788 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi037/log 2024-04-23T18:38:57.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:39:00.338 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi172/log 2024-04-23T18:39:00.339 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:39:18.640 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi173/log 2024-04-23T18:39:18.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T18:39:27.333 INFO:tasks.cephadm:Removing cluster... 2024-04-23T18:39:27.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af48a038-019c-11ef-bc93-c7b262605968 --force 2024-04-23T18:39:27.534 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:39:27.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af48a038-019c-11ef-bc93-c7b262605968 --force 2024-04-23T18:39:28.016 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:39:28.436 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af48a038-019c-11ef-bc93-c7b262605968 --force 2024-04-23T18:39:28.623 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: af48a038-019c-11ef-bc93-c7b262605968 2024-04-23T18:39:28.953 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T18:39:28.953 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:39:28.972 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:39:28.988 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T18:39:29.003 INFO:tasks.cephadm:Teardown complete 2024-04-23T18:39:29.003 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T18:39:29.015 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T18:39:29.016 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:39:29.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:39:29.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T18:39:29.147 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T18:39:29.159 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T18:39:29.160 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:39:29.163 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:39:29.165 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:39:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:39:29.177 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:39:29.179 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:39:29.180 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:39:29.180 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-23T18:39:29.180 INFO:teuthology.orchestra.run.smithi037.stdout:^- hv01.front.sepia.ceph.com 2 6 377 46 +778us[ +778us] +/- 597ms 2024-04-23T18:39:29.180 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -284us[ -266us] +/- 73ms 2024-04-23T18:39:29.180 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +341us[ +360us] +/- 87ms 2024-04-23T18:39:29.180 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:39:29.181 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:39:29.182 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-23T18:39:29.182 INFO:teuthology.orchestra.run.smithi172.stdout:^- hv01.front.sepia.ceph.com 2 6 377 47 +791us[ +791us] +/- 597ms 2024-04-23T18:39:29.182 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -278us[ -278us] +/- 73ms 2024-04-23T18:39:29.182 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +362us[ +363us] +/- 87ms 2024-04-23T18:39:29.182 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:39:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:39:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-23T18:39:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:^- hv01.front.sepia.ceph.com 2 6 377 47 +770us[ +773us] +/- 597ms 2024-04-23T18:39:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -283us[ -281us] +/- 73ms 2024-04-23T18:39:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +363us[ +363us] +/- 87ms 2024-04-23T18:39:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:39:29.184 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T18:39:29.194 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T18:39:29.195 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T18:39:29.206 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:39:29.296 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:39:29.359 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T18:39:29.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:39:29.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:39:29.452 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:39:29.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:39:29.478 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:39:29.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:39:29.505 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T18:39:29.516 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T18:39:29.528 INFO:teuthology.task.internal:Duration was 2168.107692 seconds 2024-04-23T18:39:29.529 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T18:39:29.540 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T18:39:29.540 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:39:29.546 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:39:29.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T18:39:29.580 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:39:29.583 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:39:29.585 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:39:29.909 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T18:39:29.910 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-23T18:39:29.910 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:39:29.972 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-04-23T18:39:29.972 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:39:30.003 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-04-23T18:39:30.004 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T18:39:30.033 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T18:39:30.033 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:39:30.035 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:39:30.045 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T18:39:30.097 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T18:39:30.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:39:30.101 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:39:30.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T18:39:30.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T18:39:30.492 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T18:39:30.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:39:30.517 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:39:30.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T18:39:30.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T18:39:30.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:39:30.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:39:30.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:39:30.602 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-23T18:39:30.603 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-04-23T18:39:30.629 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-04-23T18:39:30.669 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:39:30.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:39:30.683 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:39:30.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:39:30.698 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T18:39:30.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:39:30.723 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T18:39:30.736 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T18:39:30.737 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi037 2024-04-23T18:39:30.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:39:30.783 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi172 2024-04-23T18:39:30.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:39:30.818 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102/remote/smithi173 2024-04-23T18:39:30.818 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T18:39:30.854 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T18:39:30.855 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:39:30.857 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:39:30.860 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T18:39:30.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T18:39:30.916 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T18:39:30.916 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T18:39:30.928 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T18:39:30.928 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:39:30.931 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:39:30.933 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T18:39:30.943 INFO:teuthology.orchestra.run.smithi037.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:39 /home/ubuntu/cephtest 2024-04-23T18:39:30.945 INFO:teuthology.orchestra.run.smithi172.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:39 /home/ubuntu/cephtest 2024-04-23T18:39:30.958 INFO:teuthology.orchestra.run.smithi173.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 18:39 /home/ubuntu/cephtest 2024-04-23T18:39:30.959 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T18:39:30.973 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T18:39:31.003 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T18:39:31.030 DEBUG:teuthology.nuke:shortname: smithi037 2024-04-23T18:39:31.031 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:39:31.051 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 17:56:22.629390', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGineUW3WTMQEPPiuKNhabBgQQ5rdA5pJPqvef+iVH2nvXESQ3TWLfqACU/i0w0SDzyRfY5gtyUU2nFEz9E49dM='} 2024-04-23T18:39:31.058 DEBUG:teuthology.nuke:shortname: smithi172 2024-04-23T18:39:31.058 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:39:31.080 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 17:56:22.627461', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgZ6OW/XO2bJf/adqJ+8TNFlf9ZNyrFxFkFUyeYJV/xhNJl4Ry8pVDYYdidBqV1mgYTt7iwsZ3FDWcrouy59f8='} 2024-04-23T18:39:31.084 DEBUG:teuthology.nuke:shortname: smithi173 2024-04-23T18:39:31.084 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:39:31.100 INFO:teuthology.orchestra.console.smithi037:Power off 2024-04-23T18:39:31.101 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T18:39:31.114 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 17:56:22.630953', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEEKfLdJDOFNpjNjouRgn/SwbffJWlDEJkZnanyJGJ5ZpbJ8FbcQbFlY/a2uFQErQtVHZ6747PhSYVtM0YRpfU='} 2024-04-23T18:39:31.124 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-04-23T18:39:31.124 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:31.151 INFO:teuthology.orchestra.console.smithi172:Power off 2024-04-23T18:39:31.152 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T18:39:31.160 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-23T18:39:31.173 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2024-04-23T18:39:31.174 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:31.191 INFO:teuthology.orchestra.console.smithi173:Power off 2024-04-23T18:39:31.191 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T18:39:31.202 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-04-23T18:39:31.214 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2024-04-23T18:39:31.215 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:31.239 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-23T18:39:35.162 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:35.203 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:35.240 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:35.272 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-23T18:39:35.315 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-04-23T18:39:35.352 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-23T18:39:39.274 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:39.316 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:39.353 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:39.386 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-04-23T18:39:39.428 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-04-23T18:39:39.469 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-04-23T18:39:39.486 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-04-23T18:39:39.570 INFO:teuthology.orchestra.console.smithi173:Power off completed 2024-04-23T18:39:43.430 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T18:39:43.539 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2024-04-23T18:39:43.640 INFO:teuthology.orchestra.console.smithi172:Power off completed 2024-04-23T18:39:43.747 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2168.10769200325 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=155226d01bcf40c897eee7a68ed62293 status: fail success: false 2024-04-23T18:39:43.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T18:39:43.831 INFO:teuthology.run:FAIL